2024/11/21-22:59:59.559682 139783396951616 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00: fork to child time = 0.008440 sec 2024/11/21-22:59:59.560460 139783396951616 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00 2024/11/21-22:59:59.566041 139783396951616 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00 2024/11/21-22:59:59.567203 139783396951616 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/21-22:59:59.567497 139783396951616 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/21-22:59:59.623605 139783396951616 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-28153/att-00/MANIFEST-130099 2024/11/21-22:59:59.656686 139783396951616 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-28153/att-00/MANIFEST-130099 succeeded,manifest_file_number is 130099, next_file_number is 168328, last_sequence is 2690617209, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 168174 2024/11/21-22:59:59.656693 139783396951616 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/21-22:59:59.656991 139783396951616 [version_set.cc:5721] Creating manifest 168328 2024/11/21-22:59:59.691120 139783396951616 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/21-22:59:59.710286 139783396951616 [compaction_job.cc:2437] [default] [JOB 28153] Compacting 1@2 + 16@3 files to L3, score 0.82, subcompactions 1 : 1 2024/11/21-22:59:59.710301 139783396951616 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 27246 Base level 2, inputs: [168280(31MB)], [168297(40MB) 168298(166KB) 168230(1274KB) 168193(39MB) 168194(39MB) 168221(39MB) 168222(39MB) 168223(39MB) 168227(39MB) 168228(39MB) 168229(39MB) 168231(39MB) 168232(39MB) 168235(39MB) 168236(39MB) 168313(40MB)] 2024/11/21-22:59:59.717291 139783396951616 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-22:59:59.717323 139783396951616 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-22:59:59.717346 139783396951616 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-22:59:59.717351 139783396951616 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-22:59:59.717361 139783396951616 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-23:00:00.308672 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst) /dev/shm/ToplingTemp/Topling-604045-syCGkh 2024/11/21-23:00:00.477921 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst): first pass time = 0.17's, 793.228'MB/sec 2024/11/21-23:00:02.594665 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst): old prealloc_size = 147639500, real_size = 40720688 2024/11/21-23:00:02.594787 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.00 avg-zval = 312.66 2024/11/21-23:00:02.594928 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:02.752216 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168329.sst): second pass time = 2.12's, 62.229'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.567'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2196.404'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.304'MB/sec write SST data time = 0.05's, 829.552'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29248.864'MB/sec zip my value time = 2.12's, unzip length = 0.132'GB zip my value throughput = 62.229'MB/sec zip pipeline throughput = 121.681'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.00 avg-zval = 312.66 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1020.69 avg-zval = 312.66 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 68290 bytes = 221952 } seq expand size = 478030 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 7.0485 value = 3.2527 dict = 1.00 all = 3.1722 } Zip/UnZip{ index = 0.1419 value = 0.3074 dict = 1.00 all = 0.3152 } ---------------------------- total value len = 360.123308 GB avg = 0.342 KB (by entry num) total key len = 27.048906 GB avg = 0.026 KB total ukey len = 28.488871 GB avg = 0.015 KB total ukey num = 1.960409179 Billion total entry num = 1.053424884 Billion write speed all = 7.626197819 MB/sec (with seq num) write speed all = 7.460201887 MB/sec (without seq num) 2024/11/21-23:00:02.788227 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168329: 129429 keys, 42310510 bytes, temperature: kWarm 2024/11/21-23:00:02.788314 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201202788261, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168329, "file_size": 42310510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690211883, "table_properties": {"data_size": 40467808, "index_size": 220352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629385, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201200, "oldest_key_time": 0, "file_creation_time": 1732201200, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168329, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:02.840037 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst) /dev/shm/ToplingTemp/Topling-604045-59oEof 2024/11/21-23:00:03.400586 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst): first pass time = 0.56's, 239.455'MB/sec 2024/11/21-23:00:04.189425 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst): old prealloc_size = 147639500, real_size = 40359264 2024/11/21-23:00:04.189558 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.88 2024/11/21-23:00:04.189622 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:04.347553 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168330.sst): second pass time = 0.79's, 167.012'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.960'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2041.636'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.299'MB/sec write SST data time = 0.05's, 838.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 32366.883'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.012'MB/sec zip pipeline throughput = 167.012'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.01 avg-zval = 309.88 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1017.07 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 1016 bytes = 3184 } seq expand size = 7112 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0669 value = 3.2820 dict = 1.00 all = 3.2163 } Zip/UnZip{ index = 0.1415 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 360.254945 GB avg = 0.342 KB (by entry num) total key len = 27.051494 GB avg = 0.026 KB total ukey len = 28.490424 GB avg = 0.015 KB total ukey num = 1.960538607 Billion total entry num = 1.053554312 Billion write speed all = 7.628601964 MB/sec (with seq num) write speed all = 7.462590854 MB/sec (without seq num) 2024/11/21-23:00:04.389666 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168330: 129428 keys, 41730406 bytes, temperature: kWarm 2024/11/21-23:00:04.389730 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201204389697, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168330, "file_size": 41730406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690148352, "table_properties": {"data_size": 40106960, "index_size": 219776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629776, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201202, "oldest_key_time": 0, "file_creation_time": 1732201202, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168330, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:04.431378 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst) /dev/shm/ToplingTemp/Topling-604045-DNDDMd 2024/11/21-23:00:04.906678 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst): first pass time = 0.48's, 282.408'MB/sec 2024/11/21-23:00:06.938654 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst): old prealloc_size = 147639500, real_size = 40355440 2024/11/21-23:00:06.938774 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/21-23:00:06.938827 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:07.110831 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168331.sst): second pass time = 2.03's, 64.831'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 242.390'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2279.941'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.303'MB/sec write SST data time = 0.07's, 638.434'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39127.158'MB/sec zip my value time = 2.03's, unzip length = 0.132'GB zip my value throughput = 64.831'MB/sec zip pipeline throughput = 123.287'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 380 bytes = 1200 } seq expand size = 2660 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0815 value = 3.2823 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1412 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 360.386578 GB avg = 0.342 KB (by entry num) total key len = 27.054083 GB avg = 0.026 KB total ukey len = 28.491978 GB avg = 0.015 KB total ukey num = 1.960668038 Billion total entry num = 1.053683743 Billion write speed all = 7.630830340 MB/sec (with seq num) write speed all = 7.464807871 MB/sec (without seq num) 2024/11/21-23:00:07.152038 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168331: 129431 keys, 41724269 bytes, temperature: kWarm 2024/11/21-23:00:07.152129 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201207152076, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168331, "file_size": 41724269, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690190613, "table_properties": {"data_size": 40103584, "index_size": 219328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630100, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201204, "oldest_key_time": 0, "file_creation_time": 1732201204, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168331, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:07.207472 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst) /dev/shm/ToplingTemp/Topling-604045-bAQ2Df 2024/11/21-23:00:07.822326 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst): first pass time = 0.61's, 218.312'MB/sec 2024/11/21-23:00:08.624529 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst): old prealloc_size = 147639500, real_size = 40357600 2024/11/21-23:00:08.624644 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129424 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.04 avg-zval = 309.88 2024/11/21-23:00:08.624692 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:08.779074 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168332.sst): second pass time = 0.80's, 164.388'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.005'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2336.205'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 838.648'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45230.045'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.388'MB/sec zip pipeline throughput = 164.388'MB/sec entries = 129424 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.88 usrkeys = 129424 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 360.518207 GB avg = 0.342 KB (by entry num) total key len = 27.056672 GB avg = 0.026 KB total ukey len = 28.493531 GB avg = 0.015 KB total ukey num = 1.960797462 Billion total entry num = 1.053813167 Billion write speed all = 7.633223022 MB/sec (with seq num) write speed all = 7.467185617 MB/sec (without seq num) 2024/11/21-23:00:08.816237 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168332: 129424 keys, 41725396 bytes, temperature: kWarm 2024/11/21-23:00:08.816300 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201208816268, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168332, "file_size": 41725396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690204605, "table_properties": {"data_size": 40106256, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588480, "raw_average_key_size": 20, "raw_value_size": 131629476, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201207, "oldest_key_time": 0, "file_creation_time": 1732201207, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168332, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:08.856109 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst) /dev/shm/ToplingTemp/Topling-604045-mnVokd 2024/11/21-23:00:09.329262 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst): first pass time = 0.47's, 283.694'MB/sec 2024/11/21-23:00:11.428087 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst): old prealloc_size = 147639500, real_size = 40358000 2024/11/21-23:00:11.428238 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.88 2024/11/21-23:00:11.428319 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:11.619186 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168333.sst): second pass time = 2.10's, 62.770'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 218.353'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1963.003'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.229'MB/sec write SST data time = 0.05's, 837.217'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41294.029'MB/sec zip my value time = 2.10's, unzip length = 0.132'GB zip my value throughput = 62.770'MB/sec zip pipeline throughput = 123.636'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 10 bytes = 48 } seq expand size = 70 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 360.649837 GB avg = 0.342 KB (by entry num) total key len = 27.059260 GB avg = 0.026 KB total ukey len = 28.495084 GB avg = 0.015 KB total ukey num = 1.960926889 Billion total entry num = 1.053942594 Billion write speed all = 7.635439346 MB/sec (with seq num) write speed all = 7.469390836 MB/sec (without seq num) 2024/11/21-23:00:11.660255 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168333: 129427 keys, 41725509 bytes, temperature: kWarm 2024/11/21-23:00:11.660321 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201211660289, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168333, "file_size": 41725509, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690106373, "table_properties": {"data_size": 40106656, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630132, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201208, "oldest_key_time": 0, "file_creation_time": 1732201208, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168333, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:11.704265 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst) /dev/shm/ToplingTemp/Topling-604045-RvEvHe 2024/11/21-23:00:12.220806 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst): first pass time = 0.52's, 259.859'MB/sec 2024/11/21-23:00:13.019623 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst): old prealloc_size = 147639500, real_size = 40354880 2024/11/21-23:00:13.019740 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/21-23:00:13.019792 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:13.174658 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168334.sst): second pass time = 0.80's, 164.918'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 269.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2322.764'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.308'MB/sec write SST data time = 0.05's, 835.659'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38954.518'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.918'MB/sec zip pipeline throughput = 164.918'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 360.781467 GB avg = 0.342 KB (by entry num) total key len = 27.061849 GB avg = 0.026 KB total ukey len = 28.496637 GB avg = 0.015 KB total ukey num = 1.961056317 Billion total entry num = 1.054072022 Billion write speed all = 7.637848627 MB/sec (with seq num) write speed all = 7.471784813 MB/sec (without seq num) 2024/11/21-23:00:13.212752 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168334: 129428 keys, 41722733 bytes, temperature: kWarm 2024/11/21-23:00:13.212820 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201213212785, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168334, "file_size": 41722733, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2689933781, "table_properties": {"data_size": 40103536, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629269, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201211, "oldest_key_time": 0, "file_creation_time": 1732201211, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168334, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:13.251342 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst) /dev/shm/ToplingTemp/Topling-604045-tdZb8f 2024/11/21-23:00:13.789614 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst): first pass time = 0.54's, 249.366'MB/sec 2024/11/21-23:00:14.591219 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst): old prealloc_size = 147639500, real_size = 40353664 2024/11/21-23:00:14.591353 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.84 2024/11/21-23:00:14.591420 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:14.808605 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168335.sst): second pass time = 0.80's, 164.441'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 191.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2072.514'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.218'MB/sec write SST data time = 0.07's, 608.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40882.603'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.441'MB/sec zip pipeline throughput = 164.441'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 15 bytes = 48 } seq expand size = 105 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 360.913096 GB avg = 0.342 KB (by entry num) total key len = 27.064437 GB avg = 0.026 KB total ukey len = 28.498190 GB avg = 0.015 KB total ukey num = 1.961185746 Billion total entry num = 1.054201451 Billion write speed all = 7.640245962 MB/sec (with seq num) write speed all = 7.474167101 MB/sec (without seq num) 2024/11/21-23:00:14.853488 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168335: 129429 keys, 41721412 bytes, temperature: kWarm 2024/11/21-23:00:14.853577 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201214853533, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168335, "file_size": 41721412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690184265, "table_properties": {"data_size": 40102320, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629478, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201213, "oldest_key_time": 0, "file_creation_time": 1732201213, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168335, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:14.910052 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst) /dev/shm/ToplingTemp/Topling-604045-BotN0e 2024/11/21-23:00:15.493538 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst): first pass time = 0.58's, 230.052'MB/sec 2024/11/21-23:00:16.309877 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst): old prealloc_size = 147639500, real_size = 40353680 2024/11/21-23:00:16.309984 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/21-23:00:16.310033 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:16.467917 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168336.sst): second pass time = 0.81's, 161.561'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.031'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2503.558'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.299'MB/sec write SST data time = 0.05's, 839.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40818.209'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.561'MB/sec zip pipeline throughput = 161.561'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 361.044726 GB avg = 0.342 KB (by entry num) total key len = 27.067026 GB avg = 0.026 KB total ukey len = 28.499743 GB avg = 0.015 KB total ukey num = 1.961315177 Billion total entry num = 1.054330882 Billion write speed all = 7.642639318 MB/sec (with seq num) write speed all = 7.476545494 MB/sec (without seq num) 2024/11/21-23:00:16.505580 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168336: 129431 keys, 41721609 bytes, temperature: kWarm 2024/11/21-23:00:16.505645 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201216505609, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168336, "file_size": 41721609, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2689910250, "table_properties": {"data_size": 40102336, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629463, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201214, "oldest_key_time": 0, "file_creation_time": 1732201214, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168336, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:16.545369 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst) /dev/shm/ToplingTemp/Topling-604045-eYdExd 2024/11/21-23:00:16.980334 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst): first pass time = 0.43's, 308.596'MB/sec 2024/11/21-23:00:17.779682 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst): old prealloc_size = 147639500, real_size = 40355040 2024/11/21-23:00:17.779803 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.85 2024/11/21-23:00:17.779854 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:17.932150 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168337.sst): second pass time = 0.80's, 164.939'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 273.696'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2263.910'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.310'MB/sec write SST data time = 0.05's, 867.871'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39437.745'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.939'MB/sec zip pipeline throughput = 164.939'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.85 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 8 bytes = 32 } seq expand size = 56 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 361.176355 GB avg = 0.343 KB (by entry num) total key len = 27.069615 GB avg = 0.026 KB total ukey len = 28.501296 GB avg = 0.015 KB total ukey num = 1.961444608 Billion total entry num = 1.054460313 Billion write speed all = 7.645061887 MB/sec (with seq num) write speed all = 7.478952462 MB/sec (without seq num) 2024/11/21-23:00:17.969472 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168337: 129431 keys, 41722919 bytes, temperature: kWarm 2024/11/21-23:00:17.969544 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201217969503, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168337, "file_size": 41722919, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690207555, "table_properties": {"data_size": 40103696, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629279, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201216, "oldest_key_time": 0, "file_creation_time": 1732201216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168337, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:18.009475 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst) /dev/shm/ToplingTemp/Topling-604045-9WWc1e 2024/11/21-23:00:18.548231 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst): first pass time = 0.54's, 249.142'MB/sec 2024/11/21-23:00:19.336000 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst): old prealloc_size = 147639500, real_size = 40354768 2024/11/21-23:00:19.336138 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/21-23:00:19.336188 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:19.487744 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168338.sst): second pass time = 0.79's, 167.406'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1929.748'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 877.346'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 30407.007'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.406'MB/sec zip pipeline throughput = 167.406'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 361.307985 GB avg = 0.343 KB (by entry num) total key len = 27.072203 GB avg = 0.026 KB total ukey len = 28.502850 GB avg = 0.015 KB total ukey num = 1.961574038 Billion total entry num = 1.054589743 Billion write speed all = 7.647470557 MB/sec (with seq num) write speed all = 7.481345831 MB/sec (without seq num) 2024/11/21-23:00:19.526394 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168338: 129430 keys, 41722739 bytes, temperature: kWarm 2024/11/21-23:00:19.526464 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201219526423, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168338, "file_size": 41722739, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2689877483, "table_properties": {"data_size": 40103424, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629405, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201217, "oldest_key_time": 0, "file_creation_time": 1732201217, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168338, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:19.572433 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst) /dev/shm/ToplingTemp/Topling-604045-zAuzdh 2024/11/21-23:00:20.015856 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst): first pass time = 0.44's, 302.708'MB/sec 2024/11/21-23:00:20.816547 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst): old prealloc_size = 147639500, real_size = 40358448 2024/11/21-23:00:20.816659 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.88 2024/11/21-23:00:20.816713 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:20.969063 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168339.sst): second pass time = 0.80's, 164.690'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 273.609'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2398.010'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 877.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50691.130'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.690'MB/sec zip pipeline throughput = 164.690'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.88 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 361.439615 GB avg = 0.343 KB (by entry num) total key len = 27.074792 GB avg = 0.026 KB total ukey len = 28.504403 GB avg = 0.015 KB total ukey num = 1.961703464 Billion total entry num = 1.054719169 Billion write speed all = 7.649890284 MB/sec (with seq num) write speed all = 7.483750017 MB/sec (without seq num) 2024/11/21-23:00:21.008423 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168339: 129426 keys, 41726160 bytes, temperature: kWarm 2024/11/21-23:00:21.008483 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201221008450, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168339, "file_size": 41726160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690185523, "table_properties": {"data_size": 40107104, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630193, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201219, "oldest_key_time": 0, "file_creation_time": 1732201219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168339, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:21.052005 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst) /dev/shm/ToplingTemp/Topling-604045-YCYlEf 2024/11/21-23:00:21.616382 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst): first pass time = 0.56's, 237.831'MB/sec 2024/11/21-23:00:22.401240 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst): old prealloc_size = 147639500, real_size = 40356352 2024/11/21-23:00:22.401355 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.86 2024/11/21-23:00:22.401407 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:22.552305 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168340.sst): second pass time = 0.78's, 167.834'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 276.227'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2371.783'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.314'MB/sec write SST data time = 0.05's, 871.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37911.889'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.834'MB/sec zip pipeline throughput = 167.834'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 361.571245 GB avg = 0.343 KB (by entry num) total key len = 27.077380 GB avg = 0.026 KB total ukey len = 28.505956 GB avg = 0.015 KB total ukey num = 1.961832892 Billion total entry num = 1.054848597 Billion write speed all = 7.652294505 MB/sec (with seq num) write speed all = 7.486139029 MB/sec (without seq num) 2024/11/21-23:00:22.587876 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168340: 129428 keys, 41724284 bytes, temperature: kWarm 2024/11/21-23:00:22.587928 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201222587899, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168340, "file_size": 41724284, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690202112, "table_properties": {"data_size": 40105008, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629754, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201221, "oldest_key_time": 0, "file_creation_time": 1732201221, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168340, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:22.627223 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst) /dev/shm/ToplingTemp/Topling-604045-FzVUYg 2024/11/21-23:00:23.075558 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst): first pass time = 0.45's, 299.393'MB/sec 2024/11/21-23:00:23.883175 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst): old prealloc_size = 147639500, real_size = 40410016 2024/11/21-23:00:23.883304 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.98 avg-zval = 310.27 2024/11/21-23:00:23.883383 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:24.043438 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168341.sst): second pass time = 0.80's, 164.116'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.746'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2125.397'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.294'MB/sec write SST data time = 0.05's, 835.629'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39461.056'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.116'MB/sec zip pipeline throughput = 164.116'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 310.27 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.13 avg-zval = 310.27 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 2692 bytes = 7088 } seq expand size = 18844 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 7.0981 value = 3.2777 dict = 1.00 all = 3.2121 } Zip/UnZip{ index = 0.1409 value = 0.3051 dict = 1.00 all = 0.3113 } ---------------------------- total value len = 361.702894 GB avg = 0.343 KB (by entry num) total key len = 27.079969 GB avg = 0.026 KB total ukey len = 28.507509 GB avg = 0.015 KB total ukey num = 1.961962324 Billion total entry num = 1.054978029 Billion write speed all = 7.654712828 MB/sec (with seq num) write speed all = 7.488541844 MB/sec (without seq num) 2024/11/21-23:00:24.078783 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168341: 129432 keys, 41784842 bytes, temperature: kWarm 2024/11/21-23:00:24.078843 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201224078812, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168341, "file_size": 41784842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690212329, "table_properties": {"data_size": 40158672, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629725, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201222, "oldest_key_time": 0, "file_creation_time": 1732201222, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168341, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:24.117217 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst) /dev/shm/ToplingTemp/Topling-604045-q0o8if 2024/11/21-23:00:24.769850 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst): first pass time = 0.65's, 205.666'MB/sec 2024/11/21-23:00:25.580704 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst): old prealloc_size = 147639500, real_size = 40742688 2024/11/21-23:00:25.580820 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.00 avg-zval = 312.84 2024/11/21-23:00:25.580884 139783396951616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/21-23:00:25.736674 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168342.sst): second pass time = 0.81's, 162.458'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 270.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2399.588'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 832.630'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44355.906'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.458'MB/sec zip pipeline throughput = 162.458'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 312.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.57 avg-zval = 312.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 10628 bytes = 27920 } seq expand size = 74396 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 7.0980 value = 3.2508 dict = 1.00 all = 3.1852 } Zip/UnZip{ index = 0.1409 value = 0.3076 dict = 1.00 all = 0.3140 } ---------------------------- total value len = 361.834598 GB avg = 0.343 KB (by entry num) total key len = 27.082557 GB avg = 0.026 KB total ukey len = 28.509062 GB avg = 0.015 KB total ukey num = 1.962091754 Billion total entry num = 1.055107459 Billion write speed all = 7.657101638 MB/sec (with seq num) write speed all = 7.490915807 MB/sec (without seq num) 2024/11/21-23:00:25.776479 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168342: 129430 keys, 42138393 bytes, temperature: kWarm 2024/11/21-23:00:25.776546 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201225776507, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168342, "file_size": 42138393, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 2690212334, "table_properties": {"data_size": 40491344, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629916, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732201224, "oldest_key_time": 0, "file_creation_time": 1732201224, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:25.811591 139783396951616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168343.sst) /dev/shm/ToplingTemp/Topling-604045-1nraih 2024/11/21-23:00:25.821855 139783396951616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168343.sst): first pass time = 0.01's, 500.614'MB/sec 2024/11/21-23:00:25.862124 139783396951616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168343.sst): old prealloc_size = 147639500, real_size = 1587928 2024/11/21-23:00:25.862159 139783396951616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168343.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 4941 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.03 raw-val = 0.0050 GB zip-val = 0.0016 GB avg-val =1017.06 avg-zval = 321.06 2024/11/21-23:00:25.866100 139783396951616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00/168343.sst): second pass time = 0.04's, 125.992'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 414.956'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 11.532'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.485'MB/sec write SST data time = 0.00's, 1183.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.051'MB, throughput = 2410.823'MB/sec zip my value time = 0.04's, unzip length = 0.005'GB zip my value throughput = 125.992'MB/sec zip pipeline throughput = 125.992'MB/sec entries = 4941 avg-key = 20.00 avg-zkey = 0.04 avg-val = 1017.06 avg-zval = 321.06 usrkeys = 4941 avg-key = 12.00 avg-zkey = 0.04 avg-val = 1017.06 avg-zval = 321.06 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0001 GB value = 0.0050 GB dict = 0.05 MB all = 0.0051 GB } __ZipSize{ index = 0.0000 GB value = 0.0016 GB dict = 0.05 MB all = 0.0016 GB } UnZip/Zip{ index = 308.8125 value = 3.1679 dict = 1.00 all = 3.1272 } Zip/UnZip{ index = 0.0032 value = 0.3157 dict = 1.00 all = 0.3198 } ---------------------------- total value len = 361.839623 GB avg = 0.343 KB (by entry num) total key len = 27.082656 GB avg = 0.026 KB total ukey len = 28.509121 GB avg = 0.015 KB total ukey num = 1.962096695 Billion total entry num = 1.055112400 Billion write speed all = 7.657183010 MB/sec (with seq num) write speed all = 7.490996825 MB/sec (without seq num) 2024/11/21-23:00:25.872486 139783396951616 [compaction_job.cc:1948] [default] [JOB 28153] Generated table #168343: 4941 keys, 1638576 bytes, temperature: kWarm 2024/11/21-23:00:25.872539 139783396951616 EVENT_LOG_v1 {"time_micros": 1732201225872505, "cf_name": "default", "job": 28153, "event": "table_file_creation", "file_number": 168343, "file_size": 1638576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 1586336, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 98820, "raw_average_key_size": 20, "raw_value_size": 5025288, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 4941, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732201225, "oldest_key_time": 0, "file_creation_time": 1732201225, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "DRFHGKF2T4P96U3FZ4Y6", "orig_file_number": 168343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/21-23:00:25.883545 139783396951616 [compaction_job.cc:703] job-28153: subcompact[0], size: 0.586840 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28153/att-00 [168329,168330,168331,168332,168333,168334,168335,168336,168337,168338,168339,168340,168341,168342,168343] 2024/11/21-23:00:25.987004 139783396951616 (Original Log Time 2024/11/21-22:59:59.710354) EVENT_LOG_v1 {"time_micros": 1732201199710315, "job": 28153, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [168280], "files_L3": [168297, 168298, 168230, 168193, 168194, 168221, 168222, 168223, 168227, 168228, 168229, 168231, 168232, 168235, 168236, 168313], "score": 0.816029, "input_data_size": 619164319, "oldest_snapshot_seqno": 2690617121} 2024/11/21-23:00:25.987008 139783396951616 (Original Log Time 2024/11/21-23:00:25.983343) [compaction_job.cc:2022] [default] [JOB 28153] Compacted 1@2 + 16@3 files to L3 => 586839757 bytes 2024/11/21-23:00:25.987009 139783396951616 (Original Log Time 2024/11/21-23:00:25.986938) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 23.7 rd, 22.4 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(31.2, 559.2 +0.0 blob) out(559.7 +0.0 blob), read-write-amplify(36.8) write-amplify(17.9) OK, records in: 1848390, records dropped: 31445 output_compression: Snappy 2024/11/21-23:00:25.987014 139783396951616 (Original Log Time 2024/11/21-23:00:25.986987) EVENT_LOG_v1 {"time_micros": 1732201225986962, "cf": "default", "job": 28153, "event": "compaction_finished", "compaction_time_micros": 26173197, "compaction_time_cpu_micros": 4904879, "output_level": 3, "num_output_files": 15, "total_output_size": 586839757, "num_input_records": 1848390, "num_output_records": 1816945, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 377439082, "file_range_sync_nanos": 0, "file_fsync_nanos": 12420, "file_prepare_write_nanos": 25222, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/21-23:00:26.127564 139783396951616 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732156808,"starts":1732201199,"executesMs":26417,"compactionJobId":28153,"attempt":0,"compactionInputRawBytes":1916790492,"compactionInputZipBytes":619164319,"compactionOutputRawBytes":0,"compactionOutputZipBytes":586839757,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4d21f5347bdb0d463754010b3fcc2f6b-c2f55dc373e05c7d-00"}}, response = 2024/11/21-23:00:26.127665 139783396951616 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-28153/att-00: olev 3, work 26.417 s, result 6.427 ms, install 3.688 ms, input{raw 1.785 GiB zip 590.481 MiB}