2024/11/23-17:56:49.173667 139957374625344 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00: fork to child time = 0.008922 sec 2024/11/23-17:56:49.174440 139957374625344 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00 2024/11/23-17:56:49.182608 139957374625344 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00 2024/11/23-17:56:49.183610 139957374625344 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:56:49.183872 139957374625344 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:56:49.235077 139957374625344 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32496/att-00/MANIFEST-206347 2024/11/23-17:56:49.252282 139957374625344 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32496/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273521, last_sequence is 4164921847, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273385 2024/11/23-17:56:49.252289 139957374625344 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:56:49.252424 139957374625344 [version_set.cc:5721] Creating manifest 273521 2024/11/23-17:56:49.306372 139957374625344 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:56:49.340280 139957374625344 [compaction_job.cc:2437] [default] [JOB 32496] Compacting 1@2 + 19@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:56:49.340300 139957374625344 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24323 Base level 2, inputs: [273519(24MB)], [273461(37MB) 273462(37MB) 273463(39MB) 273464(38MB) 273465(36MB) 273466(34MB) 273467(34MB) 273468(38MB) 273469(36MB) 273470(35MB) 273471(34MB) 273472(35MB) 273473(35MB) 273474(36MB) 273475(33MB) 273476(34MB) 273477(36MB) 273478(37MB) 273485(36MB)] 2024/11/23-17:56:49.347251 139957374625344 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:56:49.347280 139957374625344 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:56:49.347299 139957374625344 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:56:49.347305 139957374625344 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:56:49.347315 139957374625344 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:56:49.894630 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst) /dev/shm/ToplingTemp/Topling-141015-KTYtX5 2024/11/23-17:56:50.502367 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst): first pass time = 0.61's, 220.866'MB/sec 2024/11/23-17:56:52.329865 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst): old prealloc_size = 147639500, real_size = 35504864 2024/11/23-17:56:52.330574 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927389 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.73 avg-zval = 36.13 2024/11/23-17:56:52.331493 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:56:53.898805 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273522.sst): second pass time = 1.83's, 59.264'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.57's, 25.170'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2573.578'MB/sec (index lex order gen) rebuild zvType time = 1.24's, 0.186'MB/sec write SST data time = 0.32's, 122.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37668.270'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.264'MB/sec zip pipeline throughput = 95.113'MB/sec entries = 927389 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 36.13 usrkeys = 927389 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 36.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927389 bytes = 3129952 } seq expand size = 6491723 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5133 value = 3.2305 dict = 1.00 all = 3.3988 } Zip/UnZip{ index = 0.0951 value = 0.3095 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 8.963362 GB avg = 0.168 KB (by entry num) total key len = 1.466788 GB avg = 0.028 KB total ukey len = 1.016426 GB avg = 0.015 KB total ukey num = 0.066477824 Billion total entry num = 0.053285553 Billion write speed all = 0.320911073 MB/sec (with seq num) write speed all = 0.307795309 MB/sec (without seq num) 2024/11/23-17:56:53.951794 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273522: 927389 keys, 39489562 bytes, temperature: kWarm 2024/11/23-17:56:53.951911 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355813951839, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273522, "file_size": 39489562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024708, "largest_seqno": 4164899056, "table_properties": {"data_size": 33508624, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966892, "raw_average_key_size": 28, "raw_value_size": 108250842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355809, "oldest_key_time": 0, "file_creation_time": 1732355809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:56:54.028988 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst) /dev/shm/ToplingTemp/Topling-141015-I3NTp7 2024/11/23-17:56:55.034086 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst): first pass time = 1.01's, 133.542'MB/sec 2024/11/23-17:56:56.965042 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst): old prealloc_size = 147639500, real_size = 35062992 2024/11/23-17:56:56.965857 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927373 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.73 avg-zval = 35.64 2024/11/23-17:56:56.966854 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:56:58.391803 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273523.sst): second pass time = 1.93's, 56.085'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.43's, 27.369'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2238.046'MB/sec (index lex order gen) rebuild zvType time = 1.17's, 0.198'MB/sec write SST data time = 0.26's, 153.096'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31765.851'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 56.085'MB/sec zip pipeline throughput = 77.697'MB/sec entries = 927373 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 35.64 usrkeys = 927373 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.73 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927373 bytes = 3129904 } seq expand size = 6491611 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4468 value = 3.2748 dict = 1.00 all = 3.4373 } Zip/UnZip{ index = 0.0957 value = 0.3054 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 9.078105 GB avg = 0.167 KB (by entry num) total key len = 1.492754 GB avg = 0.028 KB total ukey len = 1.034973 GB avg = 0.015 KB total ukey num = 0.067405197 Billion total entry num = 0.054212926 Billion write speed all = 0.325195415 MB/sec (with seq num) write speed all = 0.311853230 MB/sec (without seq num) 2024/11/23-17:56:58.444828 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273523: 927373 keys, 39047570 bytes, temperature: kWarm 2024/11/23-17:56:58.444920 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355818444872, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273523, "file_size": 39047570, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4164898918, "table_properties": {"data_size": 33055552, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966444, "raw_average_key_size": 28, "raw_value_size": 108251366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927373, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355814, "oldest_key_time": 0, "file_creation_time": 1732355814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:56:58.529914 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst) /dev/shm/ToplingTemp/Topling-141015-gv69O4 2024/11/23-17:56:59.781987 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst): first pass time = 1.25's, 107.201'MB/sec 2024/11/23-17:57:01.987692 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst): old prealloc_size = 147639500, real_size = 36549472 2024/11/23-17:57:01.988411 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931130 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 116.15 avg-zval = 37.10 2024/11/23-17:57:01.989341 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:03.609251 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273524.sst): second pass time = 2.20's, 49.057'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.62's, 25.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2544.331'MB/sec (index lex order gen) rebuild zvType time = 1.35's, 0.173'MB/sec write SST data time = 0.27's, 150.223'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35217.548'MB/sec zip my value time = 2.20's, unzip length = 0.108'GB zip my value throughput = 49.057'MB/sec zip pipeline throughput = 122.323'MB/sec entries = 931130 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 37.10 usrkeys = 931130 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 37.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931130 bytes = 3491760 } seq expand size = 6517910 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.5137 value = 3.1306 dict = 1.00 all = 3.2819 } Zip/UnZip{ index = 0.0951 value = 0.3194 dict = 1.00 all = 0.3047 } ---------------------------- total value len = 9.192769 GB avg = 0.167 KB (by entry num) total key len = 1.518826 GB avg = 0.028 KB total ukey len = 1.053596 GB avg = 0.015 KB total ukey num = 0.068336327 Billion total entry num = 0.055144056 Billion write speed all = 0.329472040 MB/sec (with seq num) write speed all = 0.315902876 MB/sec (without seq num) 2024/11/23-17:57:03.683634 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273524: 931130 keys, 40896150 bytes, temperature: kWarm 2024/11/23-17:57:03.683711 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355823683666, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273524, "file_size": 40896150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4164899128, "table_properties": {"data_size": 34545232, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26071640, "raw_average_key_size": 28, "raw_value_size": 108146188, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931130, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355818, "oldest_key_time": 0, "file_creation_time": 1732355818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:03.760401 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst) /dev/shm/ToplingTemp/Topling-141015-Uvwms4 2024/11/23-17:57:04.686120 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst): first pass time = 0.93's, 144.993'MB/sec 2024/11/23-17:57:06.367224 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst): old prealloc_size = 147639500, real_size = 36753072 2024/11/23-17:57:06.367786 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931515 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0347 GB avg-val = 116.09 avg-zval = 37.28 2024/11/23-17:57:06.368436 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:07.370148 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273525.sst): second pass time = 1.68's, 64.359'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 39.981'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.653'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.13's, 301.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33169.533'MB/sec zip my value time = 1.68's, unzip length = 0.108'GB zip my value throughput = 64.359'MB/sec zip pipeline throughput = 64.359'MB/sec entries = 931515 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.09 avg-zval = 37.28 usrkeys = 931515 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.78 avg-zval = 37.28 TagRS{ kinds = RS_Key0_TagN bytes = 145640 } TagArray{ size = 624723 bytes = 2342720 } seq expand size = 4373061 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.3767 value = 3.1141 dict = 1.00 all = 3.3474 } Zip/UnZip{ index = 0.0964 value = 0.3211 dict = 1.00 all = 0.2987 } ---------------------------- total value len = 9.305278 GB avg = 0.166 KB (by entry num) total key len = 1.544908 GB avg = 0.028 KB total ukey len = 1.072226 GB avg = 0.015 KB total ukey num = 0.069267842 Billion total entry num = 0.056075571 Billion write speed all = 0.333696279 MB/sec (with seq num) write speed all = 0.319899495 MB/sec (without seq num) 2024/11/23-17:57:07.451698 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273525: 931515 keys, 40096370 bytes, temperature: kWarm 2024/11/23-17:57:07.451771 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355827451735, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273525, "file_size": 40096370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34724640, "index_size": 1795392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26082420, "raw_average_key_size": 28, "raw_value_size": 108135410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931515, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355823, "oldest_key_time": 0, "file_creation_time": 1732355823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:07.492283 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst) /dev/shm/ToplingTemp/Topling-141015-yR5bx5 2024/11/23-17:57:08.418595 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst): first pass time = 0.93's, 144.901'MB/sec 2024/11/23-17:57:10.014752 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst): old prealloc_size = 147639500, real_size = 33050384 2024/11/23-17:57:10.015289 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929902 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.34 avg-zval = 33.38 2024/11/23-17:57:10.016167 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:11.051141 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273526.sst): second pass time = 1.59's, 67.833'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3473.018'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 283.960'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35254.688'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.833'MB/sec zip pipeline throughput = 67.833'MB/sec entries = 929902 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.38 usrkeys = 929902 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.34 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929902 bytes = 3487152 } seq expand size = 6509314 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4444 value = 3.4855 dict = 1.00 all = 3.5897 } Zip/UnZip{ index = 0.0957 value = 0.2869 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 9.419968 GB avg = 0.165 KB (by entry num) total key len = 1.570946 GB avg = 0.028 KB total ukey len = 1.090824 GB avg = 0.016 KB total ukey num = 0.070197744 Billion total entry num = 0.057005473 Billion write speed all = 0.337986061 MB/sec (with seq num) write speed all = 0.323962073 MB/sec (without seq num) 2024/11/23-17:57:11.098042 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273526: 929902 keys, 37389946 bytes, temperature: kWarm 2024/11/23-17:57:11.098114 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355831098078, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273526, "file_size": 37389946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405841252, "largest_seqno": 4012775902, "table_properties": {"data_size": 31037056, "index_size": 1780672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26037256, "raw_average_key_size": 28, "raw_value_size": 108180508, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929902, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355827, "oldest_key_time": 0, "file_creation_time": 1732355827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:11.135957 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst) /dev/shm/ToplingTemp/Topling-141015-pqJyc5 2024/11/23-17:57:12.078411 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst): first pass time = 0.94's, 142.424'MB/sec 2024/11/23-17:57:13.606709 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst): old prealloc_size = 147639500, real_size = 30861072 2024/11/23-17:57:13.607313 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.97 avg-zval = 30.95 2024/11/23-17:57:13.608139 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:14.654666 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273527.sst): second pass time = 1.53's, 70.775'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 33.375'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.838'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.14's, 254.520'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37064.810'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.775'MB/sec zip pipeline throughput = 70.775'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 30.95 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 30.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5130 value = 3.7469 dict = 1.00 all = 3.8376 } Zip/UnZip{ index = 0.0951 value = 0.2669 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 9.534608 GB avg = 0.165 KB (by entry num) total key len = 1.597048 GB avg = 0.028 KB total ukey len = 1.109469 GB avg = 0.016 KB total ukey num = 0.071129984 Billion total entry num = 0.057937713 Billion write speed all = 0.342276192 MB/sec (with seq num) write speed all = 0.328024441 MB/sec (without seq num) 2024/11/23-17:57:14.699182 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273527: 932240 keys, 34974750 bytes, temperature: kWarm 2024/11/23-17:57:14.699246 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355834699212, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273527, "file_size": 34974750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 28854336, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355831, "oldest_key_time": 0, "file_creation_time": 1732355831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:14.739853 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst) /dev/shm/ToplingTemp/Topling-141015-GxN5Q7 2024/11/23-17:57:15.914446 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst): first pass time = 1.17's, 114.271'MB/sec 2024/11/23-17:57:17.269906 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst): old prealloc_size = 147639500, real_size = 31626304 2024/11/23-17:57:17.270614 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925199 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 117.07 avg-zval = 32.03 2024/11/23-17:57:17.271507 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:18.310596 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273528.sst): second pass time = 1.35's, 79.958'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 34.546'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2571.660'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.21's, 173.197'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34276.219'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.958'MB/sec zip pipeline throughput = 79.958'MB/sec entries = 925199 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 32.03 usrkeys = 925199 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925199 bytes = 3469504 } seq expand size = 6476393 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5140 value = 3.6549 dict = 1.00 all = 3.7333 } Zip/UnZip{ index = 0.0951 value = 0.2736 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 9.649397 GB avg = 0.164 KB (by entry num) total key len = 1.622954 GB avg = 0.028 KB total ukey len = 1.127973 GB avg = 0.016 KB total ukey num = 0.072055183 Billion total entry num = 0.058862912 Billion write speed all = 0.346563297 MB/sec (with seq num) write speed all = 0.332085590 MB/sec (without seq num) 2024/11/23-17:57:18.355614 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273528: 925199 keys, 35951184 bytes, temperature: kWarm 2024/11/23-17:57:18.355683 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355838355647, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273528, "file_size": 35951184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775935, "table_properties": {"data_size": 29634864, "index_size": 1759936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905572, "raw_average_key_size": 28, "raw_value_size": 108312174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925199, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355834, "oldest_key_time": 0, "file_creation_time": 1732355834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:18.397368 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst) /dev/shm/ToplingTemp/Topling-141015-k0OHN3 2024/11/23-17:57:19.678671 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst): first pass time = 1.28's, 104.754'MB/sec 2024/11/23-17:57:21.265935 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst): old prealloc_size = 147639500, real_size = 35990704 2024/11/23-17:57:21.266525 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925167 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.07 avg-zval = 36.74 2024/11/23-17:57:21.268240 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:22.106105 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273529.sst): second pass time = 1.58's, 68.716'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.848'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3093.546'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 310.953'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34916.034'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.716'MB/sec zip pipeline throughput = 79.496'MB/sec entries = 925167 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 36.74 usrkeys = 925167 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 36.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925167 bytes = 3353744 } seq expand size = 6476169 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4434 value = 3.1869 dict = 1.00 all = 3.3388 } Zip/UnZip{ index = 0.0958 value = 0.3138 dict = 1.00 all = 0.2995 } ---------------------------- total value len = 9.764186 GB avg = 0.163 KB (by entry num) total key len = 1.648859 GB avg = 0.028 KB total ukey len = 1.146476 GB avg = 0.016 KB total ukey num = 0.072980350 Billion total entry num = 0.059788079 Billion write speed all = 0.350847927 MB/sec (with seq num) write speed all = 0.336144385 MB/sec (without seq num) 2024/11/23-17:57:22.156532 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273529: 925167 keys, 40199748 bytes, temperature: kWarm 2024/11/23-17:57:22.156589 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355842156558, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273529, "file_size": 40199748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33987424, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904676, "raw_average_key_size": 28, "raw_value_size": 108313086, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925167, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355838, "oldest_key_time": 0, "file_creation_time": 1732355838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273529, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:22.193372 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst) /dev/shm/ToplingTemp/Topling-141015-rihmO7 2024/11/23-17:57:23.330774 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst): first pass time = 1.14's, 118.007'MB/sec 2024/11/23-17:57:24.960732 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst): old prealloc_size = 147639500, real_size = 32997904 2024/11/23-17:57:24.961333 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928271 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.59 avg-zval = 33.40 2024/11/23-17:57:24.962216 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:25.823619 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273530.sst): second pass time = 1.63's, 66.458'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3021.838'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 288.273'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35362.615'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.458'MB/sec zip pipeline throughput = 66.458'MB/sec entries = 928271 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.40 usrkeys = 928271 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928271 bytes = 3481024 } seq expand size = 6497897 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5145 value = 3.4912 dict = 1.00 all = 3.5952 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 9.878910 GB avg = 0.163 KB (by entry num) total key len = 1.674850 GB avg = 0.028 KB total ukey len = 1.165042 GB avg = 0.016 KB total ukey num = 0.073908621 Billion total entry num = 0.060716350 Billion write speed all = 0.355133077 MB/sec (with seq num) write speed all = 0.340202954 MB/sec (without seq num) 2024/11/23-17:57:25.868846 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273530: 928271 keys, 37332344 bytes, temperature: kWarm 2024/11/23-17:57:25.868915 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355845868879, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273530, "file_size": 37332344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30999936, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991588, "raw_average_key_size": 28, "raw_value_size": 108226202, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928271, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355842, "oldest_key_time": 0, "file_creation_time": 1732355842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273530, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:25.906208 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst) /dev/shm/ToplingTemp/Topling-141015-N2zws5 2024/11/23-17:57:26.988091 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst): first pass time = 1.08's, 124.064'MB/sec 2024/11/23-17:57:28.533656 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst): old prealloc_size = 147639500, real_size = 33018784 2024/11/23-17:57:28.534223 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930517 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.24 avg-zval = 33.31 2024/11/23-17:57:28.535948 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:29.373336 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273531.sst): second pass time = 1.54's, 70.018'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.855'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.158'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 277.673'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 41308.081'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.018'MB/sec zip pipeline throughput = 70.018'MB/sec entries = 930517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.24 avg-zval = 33.31 usrkeys = 930517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.86 avg-zval = 33.31 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747520 bytes = 2803216 } seq expand size = 5232640 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3782 value = 3.4900 dict = 1.00 all = 3.6449 } Zip/UnZip{ index = 0.0964 value = 0.2865 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 9.992306 GB avg = 0.162 KB (by entry num) total key len = 1.700905 GB avg = 0.028 KB total ukey len = 1.183652 GB avg = 0.016 KB total ukey num = 0.074839138 Billion total entry num = 0.061646867 Billion write speed all = 0.359380215 MB/sec (with seq num) write speed all = 0.344222932 MB/sec (without seq num) 2024/11/23-17:57:29.420925 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273531: 930517 keys, 36823146 bytes, temperature: kWarm 2024/11/23-17:57:29.421014 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355849420959, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273531, "file_size": 36823146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30992784, "index_size": 1793216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054476, "raw_average_key_size": 28, "raw_value_size": 108163302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355845, "oldest_key_time": 0, "file_creation_time": 1732355845, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273531, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:29.461414 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst) /dev/shm/ToplingTemp/Topling-141015-njcGu7 2024/11/23-17:57:30.543843 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst): first pass time = 1.08's, 124.001'MB/sec 2024/11/23-17:57:32.133529 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst): old prealloc_size = 147639500, real_size = 30872288 2024/11/23-17:57:32.134109 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932553 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.93 avg-zval = 30.95 2024/11/23-17:57:32.134923 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:33.016217 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273532.sst): second pass time = 1.59's, 68.036'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 39.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3180.553'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 270.685'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33862.080'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.036'MB/sec zip pipeline throughput = 70.505'MB/sec entries = 932553 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.93 avg-zval = 30.95 usrkeys = 932553 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.93 avg-zval = 30.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932553 bytes = 3380512 } seq expand size = 6527871 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5127 value = 3.7453 dict = 1.00 all = 3.8235 } Zip/UnZip{ index = 0.0951 value = 0.2670 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 10.106940 GB avg = 0.162 KB (by entry num) total key len = 1.727016 GB avg = 0.028 KB total ukey len = 1.202303 GB avg = 0.016 KB total ukey num = 0.075771691 Billion total entry num = 0.062579420 Billion write speed all = 0.363665188 MB/sec (with seq num) write speed all = 0.348280338 MB/sec (without seq num) 2024/11/23-17:57:33.062576 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273532: 932553 keys, 35103318 bytes, temperature: kWarm 2024/11/23-17:57:33.062666 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355853062617, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273532, "file_size": 35103318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 28864848, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26111484, "raw_average_key_size": 28, "raw_value_size": 108106270, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355849, "oldest_key_time": 0, "file_creation_time": 1732355849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273532, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:33.133609 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst) /dev/shm/ToplingTemp/Topling-141015-ArNkp5 2024/11/23-17:57:34.267309 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst): first pass time = 1.13's, 118.395'MB/sec 2024/11/23-17:57:35.916811 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst): old prealloc_size = 147639500, real_size = 32608240 2024/11/23-17:57:35.917352 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931936 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.02 avg-zval = 32.82 2024/11/23-17:57:35.918158 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:36.772242 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273533.sst): second pass time = 1.65's, 65.601'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.064'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.000'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 291.012'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29520.811'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.601'MB/sec zip pipeline throughput = 104.705'MB/sec entries = 931936 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.02 avg-zval = 32.82 usrkeys = 931936 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.02 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931936 bytes = 3378288 } seq expand size = 6523552 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4455 value = 3.5345 dict = 1.00 all = 3.6435 } Zip/UnZip{ index = 0.0957 value = 0.2829 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 10.221588 GB avg = 0.161 KB (by entry num) total key len = 1.753110 GB avg = 0.028 KB total ukey len = 1.220942 GB avg = 0.016 KB total ukey num = 0.076703627 Billion total entry num = 0.063511356 Billion write speed all = 0.367947791 MB/sec (with seq num) write speed all = 0.352335631 MB/sec (without seq num) 2024/11/23-17:57:36.817757 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273533: 931936 keys, 36837476 bytes, temperature: kWarm 2024/11/23-17:57:36.817825 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355856817790, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273533, "file_size": 36837476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3518605479, "largest_seqno": 4012776268, "table_properties": {"data_size": 30590688, "index_size": 1784384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26094208, "raw_average_key_size": 28, "raw_value_size": 108123604, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931936, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355853, "oldest_key_time": 0, "file_creation_time": 1732355853, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273533, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:36.856013 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst) /dev/shm/ToplingTemp/Topling-141015-Lbne43 2024/11/23-17:57:37.700626 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst): first pass time = 0.84's, 158.917'MB/sec 2024/11/23-17:57:39.024750 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst): old prealloc_size = 147639500, real_size = 33953104 2024/11/23-17:57:39.025330 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922525 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.49 avg-zval = 34.64 2024/11/23-17:57:39.026188 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:39.870880 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273534.sst): second pass time = 1.32's, 81.905'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.232'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.933'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 297.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34455.725'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.905'MB/sec zip pipeline throughput = 81.905'MB/sec entries = 922525 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.49 avg-zval = 34.64 usrkeys = 922525 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.49 avg-zval = 34.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922525 bytes = 3459488 } seq expand size = 6457675 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4426 value = 3.3918 dict = 1.00 all = 3.5070 } Zip/UnZip{ index = 0.0958 value = 0.2948 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 10.336432 GB avg = 0.160 KB (by entry num) total key len = 1.778941 GB avg = 0.028 KB total ukey len = 1.239392 GB avg = 0.016 KB total ukey num = 0.077626152 Billion total entry num = 0.064433881 Billion write speed all = 0.372234899 MB/sec (with seq num) write speed all = 0.356397475 MB/sec (without seq num) 2024/11/23-17:57:39.917088 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273534: 922525 keys, 38271144 bytes, temperature: kWarm 2024/11/23-17:57:39.917158 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355859917122, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273534, "file_size": 38271144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 31955456, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830700, "raw_average_key_size": 28, "raw_value_size": 108387078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355856, "oldest_key_time": 0, "file_creation_time": 1732355856, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273534, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:39.958210 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst) /dev/shm/ToplingTemp/Topling-141015-fuEOO7 2024/11/23-17:57:40.986269 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst): first pass time = 1.03's, 130.559'MB/sec 2024/11/23-17:57:42.323017 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst): old prealloc_size = 147639500, real_size = 33708800 2024/11/23-17:57:42.323599 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932773 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0317 GB avg-val = 115.89 avg-zval = 33.99 2024/11/23-17:57:42.324400 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:43.174576 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273535.sst): second pass time = 1.34's, 80.916'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.001'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 299.913'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36196.023'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.916'MB/sec zip pipeline throughput = 80.916'MB/sec entries = 932773 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.89 avg-zval = 33.99 usrkeys = 932773 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.89 avg-zval = 33.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932773 bytes = 3381312 } seq expand size = 6529411 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5141 value = 3.4100 dict = 1.00 all = 3.5376 } Zip/UnZip{ index = 0.0951 value = 0.2933 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 10.451062 GB avg = 0.160 KB (by entry num) total key len = 1.805059 GB avg = 0.028 KB total ukey len = 1.258048 GB avg = 0.016 KB total ukey num = 0.078558925 Billion total entry num = 0.065366654 Billion write speed all = 0.376521026 MB/sec (with seq num) write speed all = 0.360455962 MB/sec (without seq num) 2024/11/23-17:57:43.222287 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273535: 932773 keys, 37940428 bytes, temperature: kWarm 2024/11/23-17:57:43.222354 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355863222318, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273535, "file_size": 37940428, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 31701104, "index_size": 1774336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26117644, "raw_average_key_size": 28, "raw_value_size": 108100226, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355859, "oldest_key_time": 0, "file_creation_time": 1732355859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273535, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:43.256551 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst) /dev/shm/ToplingTemp/Topling-141015-4GvYI3 2024/11/23-17:57:44.232549 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst): first pass time = 0.98's, 137.523'MB/sec 2024/11/23-17:57:45.543381 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst): old prealloc_size = 147639500, real_size = 32686880 2024/11/23-17:57:45.543936 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926281 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.90 avg-zval = 33.14 2024/11/23-17:57:45.544774 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:46.383046 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273536.sst): second pass time = 1.31's, 82.657'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.084'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.174'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 294.245'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33944.015'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.657'MB/sec zip pipeline throughput = 82.657'MB/sec entries = 926281 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 33.14 usrkeys = 926281 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926281 bytes = 3473568 } seq expand size = 6483967 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5125 value = 3.5279 dict = 1.00 all = 3.6260 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 10.565828 GB avg = 0.159 KB (by entry num) total key len = 1.830995 GB avg = 0.028 KB total ukey len = 1.276573 GB avg = 0.016 KB total ukey num = 0.079485206 Billion total entry num = 0.066292935 Billion write speed all = 0.380805998 MB/sec (with seq num) write speed all = 0.364514889 MB/sec (without seq num) 2024/11/23-17:57:46.429476 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273536: 926281 keys, 37015110 bytes, temperature: kWarm 2024/11/23-17:57:46.429545 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355866429510, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273536, "file_size": 37015110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 30692880, "index_size": 1762240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935868, "raw_average_key_size": 28, "raw_value_size": 108281922, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355863, "oldest_key_time": 0, "file_creation_time": 1732355863, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273536, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:46.469818 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst) /dev/shm/ToplingTemp/Topling-141015-CETlH6 2024/11/23-17:57:47.488459 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst): first pass time = 1.02's, 131.768'MB/sec 2024/11/23-17:57:48.808103 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst): old prealloc_size = 147639500, real_size = 32953136 2024/11/23-17:57:48.808687 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924947 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.11 avg-zval = 33.45 2024/11/23-17:57:48.810436 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:49.656350 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273537.sst): second pass time = 1.32's, 82.132'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3173.900'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 295.634'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33025.395'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.132'MB/sec zip pipeline throughput = 82.132'MB/sec entries = 924947 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.11 avg-zval = 33.45 usrkeys = 924947 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.11 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924947 bytes = 3468560 } seq expand size = 6474629 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3749 value = 3.5011 dict = 1.00 all = 3.6006 } Zip/UnZip{ index = 0.0964 value = 0.2856 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 10.680622 GB avg = 0.159 KB (by entry num) total key len = 1.856893 GB avg = 0.028 KB total ukey len = 1.295072 GB avg = 0.016 KB total ukey num = 0.080410153 Billion total entry num = 0.067217882 Billion write speed all = 0.385089070 MB/sec (with seq num) write speed all = 0.368572322 MB/sec (without seq num) 2024/11/23-17:57:49.701465 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273537: 924947 keys, 37276930 bytes, temperature: kWarm 2024/11/23-17:57:49.701535 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355869701499, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273537, "file_size": 37276930, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30938656, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898516, "raw_average_key_size": 28, "raw_value_size": 108319226, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924947, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355866, "oldest_key_time": 0, "file_creation_time": 1732355866, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273537, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:49.743318 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst) /dev/shm/ToplingTemp/Topling-141015-UJ0Ck5 2024/11/23-17:57:50.754792 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst): first pass time = 1.01's, 132.701'MB/sec 2024/11/23-17:57:52.190757 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst): old prealloc_size = 147639500, real_size = 36006864 2024/11/23-17:57:52.191392 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930405 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.26 avg-zval = 36.55 2024/11/23-17:57:52.191720 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:52.930748 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273538.sst): second pass time = 1.44's, 75.372'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 51.211'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2905.028'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.379'MB/sec write SST data time = 0.13's, 299.761'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32711.261'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.372'MB/sec zip pipeline throughput = 75.372'MB/sec entries = 930405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 36.55 usrkeys = 930405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.31 avg-zval = 36.55 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 272649 bytes = 886128 } seq expand size = 1908543 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5124 value = 3.1810 dict = 1.00 all = 3.5419 } Zip/UnZip{ index = 0.0951 value = 0.3144 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 10.790697 GB avg = 0.158 KB (by entry num) total key len = 1.882945 GB avg = 0.028 KB total ukey len = 1.313680 GB avg = 0.016 KB total ukey num = 0.081340558 Billion total entry num = 0.068148287 Billion write speed all = 0.389231039 MB/sec (with seq num) write speed all = 0.372487356 MB/sec (without seq num) 2024/11/23-17:57:52.976987 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273538: 930405 keys, 37894693 bytes, temperature: kWarm 2024/11/23-17:57:52.977060 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355872977023, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273538, "file_size": 37894693, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 34003968, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051340, "raw_average_key_size": 28, "raw_value_size": 108166522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355869, "oldest_key_time": 0, "file_creation_time": 1732355869, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273538, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:53.014681 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst) /dev/shm/ToplingTemp/Topling-141015-sYerH4 2024/11/23-17:57:53.960350 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst): first pass time = 0.95's, 141.935'MB/sec 2024/11/23-17:57:55.441829 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst): old prealloc_size = 147639500, real_size = 37285840 2024/11/23-17:57:55.442446 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925707 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 116.99 avg-zval = 38.11 2024/11/23-17:57:55.442802 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:56.171718 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273539.sst): second pass time = 1.48's, 73.163'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.909'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2976.732'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.383'MB/sec write SST data time = 0.12's, 315.904'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32108.833'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.163'MB/sec zip pipeline throughput = 73.163'MB/sec entries = 925707 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 38.11 usrkeys = 925707 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.12 avg-zval = 38.11 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 281870 bytes = 1057024 } seq expand size = 1973090 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4461 value = 3.0695 dict = 1.00 all = 3.4112 } Zip/UnZip{ index = 0.0957 value = 0.3258 dict = 1.00 all = 0.2932 } ---------------------------- total value len = 10.900968 GB avg = 0.158 KB (by entry num) total key len = 1.908864 GB avg = 0.028 KB total ukey len = 1.332195 GB avg = 0.016 KB total ukey num = 0.082266265 Billion total entry num = 0.069073994 Billion write speed all = 0.393374557 MB/sec (with seq num) write speed all = 0.376405122 MB/sec (without seq num) 2024/11/23-17:57:56.218887 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273539: 925707 keys, 39346205 bytes, temperature: kWarm 2024/11/23-17:57:56.218959 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355876218922, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273539, "file_size": 39346205, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4153652930, "table_properties": {"data_size": 35281856, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919796, "raw_average_key_size": 28, "raw_value_size": 108297942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355873, "oldest_key_time": 0, "file_creation_time": 1732355873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273539, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:56.263018 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst) /dev/shm/ToplingTemp/Topling-141015-4gmDI5 2024/11/23-17:57:57.321510 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst): first pass time = 1.06's, 126.806'MB/sec 2024/11/23-17:57:58.667696 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst): old prealloc_size = 147639500, real_size = 35425008 2024/11/23-17:57:58.668271 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927930 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.64 avg-zval = 36.02 2024/11/23-17:57:58.669148 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:57:59.503855 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273540.sst): second pass time = 1.35's, 80.470'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.995'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3193.059'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 311.543'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34776.972'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.470'MB/sec zip pipeline throughput = 80.470'MB/sec entries = 927930 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 36.02 usrkeys = 927930 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 36.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927930 bytes = 3015792 } seq expand size = 6495510 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5141 value = 3.2379 dict = 1.00 all = 3.4157 } Zip/UnZip{ index = 0.0951 value = 0.3088 dict = 1.00 all = 0.2928 } ---------------------------- total value len = 11.015699 GB avg = 0.157 KB (by entry num) total key len = 1.934846 GB avg = 0.028 KB total ukey len = 1.350753 GB avg = 0.016 KB total ukey num = 0.083194195 Billion total entry num = 0.070001924 Billion write speed all = 0.397655001 MB/sec (with seq num) write speed all = 0.380459361 MB/sec (without seq num) 2024/11/23-17:57:59.551121 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273540: 927930 keys, 39294184 bytes, temperature: kWarm 2024/11/23-17:57:59.551190 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355879551155, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273540, "file_size": 39294184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4164899032, "table_properties": {"data_size": 33427744, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982040, "raw_average_key_size": 28, "raw_value_size": 108235700, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927930, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355876, "oldest_key_time": 0, "file_creation_time": 1732355876, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273540, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:57:59.587494 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst) /dev/shm/ToplingTemp/Topling-141015-LuJed4 2024/11/23-17:57:59.675716 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst): first pass time = 0.09's, 238.278'MB/sec 2024/11/23-17:57:59.865428 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst): old prealloc_size = 147639500, real_size = 5886224 2024/11/23-17:57:59.865569 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 146592 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0169 GB zip-val = 0.0056 GB avg-val = 115.34 avg-zval = 37.98 2024/11/23-17:57:59.865749 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/23-17:57:59.996109 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00/273541.sst): second pass time = 0.19's, 89.449'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 49.651'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2362.409'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.333'MB/sec write SST data time = 0.02's, 314.736'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.178'MB, throughput = 6580.643'MB/sec zip my value time = 0.19's, unzip length = 0.017'GB zip my value throughput = 89.449'MB/sec zip pipeline throughput = 89.449'MB/sec entries = 146592 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.34 avg-zval = 37.98 usrkeys = 146592 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.34 avg-zval = 37.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 146592 bytes = 458112 } seq expand size = 1026144 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0169 GB dict = 0.18 MB all = 0.0210 GB } __ZipSize{ index = 0.0003 GB value = 0.0056 GB dict = 0.18 MB all = 0.0065 GB } UnZip/Zip{ index = 10.3948 value = 3.0370 dict = 1.00 all = 3.2389 } Zip/UnZip{ index = 0.0962 value = 0.3293 dict = 1.00 all = 0.3087 } ---------------------------- total value len = 11.033633 GB avg = 0.157 KB (by entry num) total key len = 1.938951 GB avg = 0.028 KB total ukey len = 1.353685 GB avg = 0.016 KB total ukey num = 0.083340787 Billion total entry num = 0.070148516 Billion write speed all = 0.398325695 MB/sec (with seq num) write speed all = 0.381094306 MB/sec (without seq num) 2024/11/23-17:58:00.011506 139957374625344 [compaction_job.cc:1948] [default] [JOB 32496] Generated table #273541: 146592 keys, 6487541 bytes, temperature: kWarm 2024/11/23-17:58:00.011567 139957374625344 EVENT_LOG_v1 {"time_micros": 1732355880011534, "cf_name": "default", "job": 32496, "event": "table_file_creation", "file_number": 273541, "file_size": 6487541, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4145861985, "largest_seqno": 4163385207, "table_properties": {"data_size": 5567360, "index_size": 282048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4104576, "raw_average_key_size": 28, "raw_value_size": 16908044, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 146592, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355879, "oldest_key_time": 0, "file_creation_time": 1732355879, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273541, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:58:00.023509 139957374625344 [compaction_job.cc:703] job-32496: subcompact[0], size: 0.727668 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32496/att-00 [273522,273523,273524,273525,273526,273527,273528,273529,273530,273531,273532,273533,273534,273535,273536,273537,273538,273539,273540,273541] 2024/11/23-17:58:00.172564 139957374625344 (Original Log Time 2024/11/23-17:56:49.340366) EVENT_LOG_v1 {"time_micros": 1732355809340317, "job": 32496, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273519], "files_L3": [273461, 273462, 273463, 273464, 273465, 273466, 273467, 273468, 273469, 273470, 273471, 273472, 273473, 273474, 273475, 273476, 273477, 273478, 273485], "score": 0.992637, "input_data_size": 748347728, "oldest_snapshot_seqno": 4164921820} 2024/11/23-17:58:00.172574 139957374625344 (Original Log Time 2024/11/23-17:58:00.170957) [compaction_job.cc:2022] [default] [JOB 32496] Compacted 1@2 + 19@3 files to L3 => 727667799 bytes 2024/11/23-17:58:00.172575 139957374625344 (Original Log Time 2024/11/23-17:58:00.172490) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 10.6 rd, 10.3 wr, level 3, files in(1, 19) out(20 +0 blob) MB in(25.0, 688.7 +0.0 blob) out(694.0 +0.0 blob), read-write-amplify(56.4) write-amplify(27.8) OK, records in: 17859638, records dropped: 69286 output_compression: Snappy 2024/11/23-17:58:00.172579 139957374625344 (Original Log Time 2024/11/23-17:58:00.172535) EVENT_LOG_v1 {"time_micros": 1732355880172514, "cf": "default", "job": 32496, "event": "compaction_finished", "compaction_time_micros": 70683151, "compaction_time_cpu_micros": 29548169, "output_level": 3, "num_output_files": 20, "total_output_size": 727667799, "num_input_records": 17859638, "num_output_records": 17790352, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 579344986, "file_range_sync_nanos": 0, "file_fsync_nanos": 15424, "file_prepare_write_nanos": 55732, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-17:58:00.305337 139957374625344 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732355809,"executesMs":70989,"compactionJobId":32496,"attempt":0,"compactionInputRawBytes":2580989140,"compactionInputZipBytes":748347728,"compactionOutputRawBytes":0,"compactionOutputZipBytes":727667799,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6268c17f2d6f903882aca09ffae241e4-de08b9336ddbd0af-00"}}, response = 2024/11/23-17:58:00.305433 139957374625344 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32496/att-00: olev 3, work 70.989 s, result 7.761 ms, install 1.636 ms, input{raw 2.404 GiB zip 713.680 MiB}