2024/11/22-14:58:28.248621 140025037178432 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00: fork to child time = 0.010530 sec 2024/11/22-14:58:28.249507 140025037178432 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00 2024/11/22-14:58:28.256703 140025037178432 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00 2024/11/22-14:58:28.257836 140025037178432 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-14:58:28.258118 140025037178432 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-14:58:28.329351 140025037178432 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01824/att-00/MANIFEST-206347 2024/11/22-14:58:28.343945 140025037178432 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01824/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 209477, last_sequence is 3310124116, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 209326 2024/11/22-14:58:28.343954 140025037178432 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-14:58:28.344214 140025037178432 [version_set.cc:5721] Creating manifest 209477 2024/11/22-14:58:28.376510 140025037178432 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-14:58:28.395402 140025037178432 [compaction_job.cc:2437] [default] [JOB 1824] Compacting 1@2 + 21@3 files to L3, score 0.90, subcompactions 1 : 1 2024/11/22-14:58:28.395425 140025037178432 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 1496 Base level 2, inputs: [209473(12MB)], [209401(40MB) 209402(39MB) 209403(39MB) 209422(40MB) 209426(39MB) 209424(39MB) 209425(39MB) 209423(39MB) 209427(39MB) 209433(39MB) 209434(39MB) 209435(39MB) 209436(39MB) 209437(39MB) 209438(39MB) 209439(39MB) 209440(39MB) 209441(39MB) 209442(39MB) 209444(39MB) 209443(17MB)] 2024/11/22-14:58:28.399846 140025037178432 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:58:28.399878 140025037178432 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:58:28.399898 140025037178432 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:58:28.399904 140025037178432 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:58:28.399915 140025037178432 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:58:28.802206 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst) /dev/shm/ToplingTemp/Topling-159797-SIju6h 2024/11/22-14:58:29.275871 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst): first pass time = 0.47's, 283.384'MB/sec 2024/11/22-14:58:30.153186 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst): old prealloc_size = 147639500, real_size = 40754976 2024/11/22-14:58:30.153305 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.02 avg-zval = 312.93 2024/11/22-14:58:30.153435 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:30.328005 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209478.sst): second pass time = 0.88's, 150.144'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 242.482'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2286.781'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.299'MB/sec write SST data time = 0.07's, 637.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48540.437'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.144'MB/sec zip pipeline throughput = 150.144'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.02 avg-zval = 312.93 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1021.22 avg-zval = 312.93 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 77789 bytes = 262560 } seq expand size = 544523 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 7.0443 value = 3.2500 dict = 1.00 all = 3.1666 } Zip/UnZip{ index = 0.1420 value = 0.3077 dict = 1.00 all = 0.3158 } ---------------------------- total value len = 0.132174 GB avg = 1.021 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129427 Billion total entry num = 0.000129427 Billion write speed all = 0.006134703 MB/sec (with seq num) write speed all = 0.006087568 MB/sec (without seq num) 2024/11/22-14:58:30.374989 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209478: 129427 keys, 42385294 bytes, temperature: kWarm 2024/11/22-14:58:30.375101 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258710375030, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209478, "file_size": 42385294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310092480, "table_properties": {"data_size": 40501968, "index_size": 220480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629378, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258708, "oldest_key_time": 0, "file_creation_time": 1732258708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:30.426232 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst) /dev/shm/ToplingTemp/Topling-159797-9J6B2h 2024/11/22-14:58:30.701172 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst): first pass time = 0.27's, 488.266'MB/sec 2024/11/22-14:58:31.554361 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst): old prealloc_size = 147639500, real_size = 40374720 2024/11/22-14:58:31.554491 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.96 2024/11/22-14:58:31.554544 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:31.711127 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209479.sst): second pass time = 0.85's, 154.391'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.390'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2046.655'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.305'MB/sec write SST data time = 0.05's, 828.220'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 32459.439'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.391'MB/sec zip pipeline throughput = 154.391'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.98 avg-zval = 309.96 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1017.21 avg-zval = 309.96 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 4182 bytes = 13600 } seq expand size = 29274 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 6.9697 value = 3.2810 dict = 1.00 all = 3.2143 } Zip/UnZip{ index = 0.1435 value = 0.3048 dict = 1.00 all = 0.3111 } ---------------------------- total value len = 0.263833 GB avg = 1.019 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258859 Billion total entry num = 0.000258859 Billion write speed all = 0.012245203 MB/sec (with seq num) write speed all = 0.012150938 MB/sec (without seq num) 2024/11/22-14:58:31.750299 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209479: 129432 keys, 41756630 bytes, temperature: kWarm 2024/11/22-14:58:31.750356 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258711750325, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209479, "file_size": 41756630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310088235, "table_properties": {"data_size": 40119344, "index_size": 222848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629789, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258710, "oldest_key_time": 0, "file_creation_time": 1732258710, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:31.787927 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst) /dev/shm/ToplingTemp/Topling-159797-GkrpUh 2024/11/22-14:58:32.285304 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst): first pass time = 0.50's, 269.872'MB/sec 2024/11/22-14:58:33.527027 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst): old prealloc_size = 147639500, real_size = 40545808 2024/11/22-14:58:33.527143 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.03 avg-zval = 311.33 2024/11/22-14:58:33.527204 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:33.681790 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209480.sst): second pass time = 1.23's, 106.947'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2324.685'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.304'MB/sec write SST data time = 0.05's, 866.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39887.371'MB/sec zip my value time = 1.23's, unzip length = 0.132'GB zip my value throughput = 106.947'MB/sec zip pipeline throughput = 180.979'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.03 avg-zval = 311.33 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1017.06 avg-zval = 311.33 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 542 bytes = 1712 } seq expand size = 3794 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 7.0750 value = 3.2667 dict = 1.00 all = 3.2021 } Zip/UnZip{ index = 0.1413 value = 0.3061 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 0.395466 GB avg = 1.018 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388285 Billion total entry num = 0.000388285 Billion write speed all = 0.018353270 MB/sec (with seq num) write speed all = 0.018211886 MB/sec (without seq num) 2024/11/22-14:58:33.720592 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209480: 129426 keys, 41915563 bytes, temperature: kWarm 2024/11/22-14:58:33.720657 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258713720625, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209480, "file_size": 41915563, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310069610, "table_properties": {"data_size": 40293760, "index_size": 219520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629627, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258711, "oldest_key_time": 0, "file_creation_time": 1732258711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:33.762646 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst) /dev/shm/ToplingTemp/Topling-159797-YypRfk 2024/11/22-14:58:34.207225 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst): first pass time = 0.44's, 301.927'MB/sec 2024/11/22-14:58:35.618654 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst): old prealloc_size = 147639500, real_size = 40600032 2024/11/22-14:58:35.618766 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.74 2024/11/22-14:58:35.618825 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:35.776605 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209481.sst): second pass time = 1.41's, 93.351'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.721'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2438.306'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.306'MB/sec write SST data time = 0.05's, 803.048'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41443.154'MB/sec zip my value time = 1.41's, unzip length = 0.132'GB zip my value throughput = 93.351'MB/sec zip pipeline throughput = 189.530'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 311.74 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 311.74 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 7.0980 value = 3.2623 dict = 1.00 all = 3.1981 } Zip/UnZip{ index = 0.1409 value = 0.3065 dict = 1.00 all = 0.3127 } ---------------------------- total value len = 0.527096 GB avg = 1.018 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517715 Billion total entry num = 0.000517715 Billion write speed all = 0.024459924 MB/sec (with seq num) write speed all = 0.024271430 MB/sec (without seq num) 2024/11/22-14:58:35.840229 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209481: 129430 keys, 41967661 bytes, temperature: kWarm 2024/11/22-14:58:35.840287 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258715840257, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209481, "file_size": 41967661, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309802572, "table_properties": {"data_size": 40348688, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629294, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258713, "oldest_key_time": 0, "file_creation_time": 1732258713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:35.877624 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst) /dev/shm/ToplingTemp/Topling-159797-mhg29g 2024/11/22-14:58:36.429654 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst): first pass time = 0.55's, 243.151'MB/sec 2024/11/22-14:58:37.236410 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst): old prealloc_size = 147639500, real_size = 40356256 2024/11/22-14:58:37.236530 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.86 2024/11/22-14:58:37.236589 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:37.390322 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209482.sst): second pass time = 0.81's, 163.292'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2281.329'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.312'MB/sec write SST data time = 0.05's, 832.856'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39932.447'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.292'MB/sec zip pipeline throughput = 163.292'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.658725 GB avg = 1.018 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647143 Billion total entry num = 0.000647143 Billion write speed all = 0.030566078 MB/sec (with seq num) write speed all = 0.030330478 MB/sec (without seq num) 2024/11/22-14:58:37.428153 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209482: 129428 keys, 41723892 bytes, temperature: kWarm 2024/11/22-14:58:37.428211 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258717428181, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209482, "file_size": 41723892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310002998, "table_properties": {"data_size": 40104912, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629241, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258715, "oldest_key_time": 0, "file_creation_time": 1732258715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:37.466672 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst) /dev/shm/ToplingTemp/Topling-159797-uDrBMk 2024/11/22-14:58:38.019210 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst): first pass time = 0.55's, 242.931'MB/sec 2024/11/22-14:58:39.109380 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst): old prealloc_size = 147639500, real_size = 40356720 2024/11/22-14:58:39.109497 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.86 2024/11/22-14:58:39.109566 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:39.263337 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209483.sst): second pass time = 1.09's, 120.826'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.066'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2311.844'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 850.939'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34459.049'MB/sec zip my value time = 1.09's, unzip length = 0.132'GB zip my value throughput = 120.826'MB/sec zip pipeline throughput = 190.928'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 10 bytes = 32 } seq expand size = 70 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.790355 GB avg = 1.018 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776572 Billion total entry num = 0.000776572 Billion write speed all = 0.036670925 MB/sec (with seq num) write speed all = 0.036388229 MB/sec (without seq num) 2024/11/22-14:58:39.305876 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209483: 129429 keys, 41724409 bytes, temperature: kWarm 2024/11/22-14:58:39.305956 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258719305912, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209483, "file_size": 41724409, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310037319, "table_properties": {"data_size": 40105376, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629734, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258717, "oldest_key_time": 0, "file_creation_time": 1732258717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:39.378655 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst) /dev/shm/ToplingTemp/Topling-159797-ytMdJg 2024/11/22-14:58:39.920761 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst): first pass time = 0.54's, 247.611'MB/sec 2024/11/22-14:58:40.793497 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst): old prealloc_size = 147639500, real_size = 40356800 2024/11/22-14:58:40.793614 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.87 2024/11/22-14:58:40.793662 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:40.947017 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209484.sst): second pass time = 0.87's, 150.920'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.818'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2359.609'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 846.569'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44464.598'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 150.920'MB/sec zip pipeline throughput = 150.920'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.921985 GB avg = 1.018 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000905999 Billion total entry num = 0.000905999 Billion write speed all = 0.042775103 MB/sec (with seq num) write speed all = 0.042445317 MB/sec (without seq num) 2024/11/22-14:58:40.984634 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209484: 129427 keys, 41725006 bytes, temperature: kWarm 2024/11/22-14:58:40.984700 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258720984666, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209484, "file_size": 41725006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310046966, "table_properties": {"data_size": 40105456, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629798, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258719, "oldest_key_time": 0, "file_creation_time": 1732258719, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:41.021051 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst) /dev/shm/ToplingTemp/Topling-159797-6ywH4h 2024/11/22-14:58:41.668047 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst): first pass time = 0.65's, 207.463'MB/sec 2024/11/22-14:58:42.519532 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst): old prealloc_size = 147639500, real_size = 40355088 2024/11/22-14:58:42.519649 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.85 2024/11/22-14:58:42.519707 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:42.689093 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209485.sst): second pass time = 0.85's, 154.886'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2373.250'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.301'MB/sec write SST data time = 0.06's, 674.194'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45884.827'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.886'MB/sec zip pipeline throughput = 154.886'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.053615 GB avg = 1.018 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035429 Billion total entry num = 0.001035429 Billion write speed all = 0.048878232 MB/sec (with seq num) write speed all = 0.048501363 MB/sec (without seq num) 2024/11/22-14:58:42.729593 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209485: 129430 keys, 41722890 bytes, temperature: kWarm 2024/11/22-14:58:42.729677 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258722729633, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209485, "file_size": 41722890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310067656, "table_properties": {"data_size": 40103744, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630115, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258721, "oldest_key_time": 0, "file_creation_time": 1732258721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209485, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:42.778763 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst) /dev/shm/ToplingTemp/Topling-159797-qtWTGg 2024/11/22-14:58:43.544831 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst): first pass time = 0.77's, 175.220'MB/sec 2024/11/22-14:58:44.360076 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst): old prealloc_size = 147639500, real_size = 40352144 2024/11/22-14:58:44.360189 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129436 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.95 avg-zval = 309.81 2024/11/22-14:58:44.360261 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:44.512885 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209486.sst): second pass time = 0.81's, 161.578'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 273.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2484.654'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 879.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37856.433'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.578'MB/sec zip pipeline throughput = 161.578'MB/sec entries = 129436 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.81 usrkeys = 129436 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2825 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.185245 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164865 Billion total entry num = 0.001164865 Billion write speed all = 0.054980189 MB/sec (with seq num) write speed all = 0.054556243 MB/sec (without seq num) 2024/11/22-14:58:44.551602 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209486: 129436 keys, 41719899 bytes, temperature: kWarm 2024/11/22-14:58:44.551660 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258724551628, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209486, "file_size": 41719899, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310065957, "table_properties": {"data_size": 40100800, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588720, "raw_average_key_size": 20, "raw_value_size": 131629989, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258722, "oldest_key_time": 0, "file_creation_time": 1732258722, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209486, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:44.587018 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst) /dev/shm/ToplingTemp/Topling-159797-biBbDj 2024/11/22-14:58:45.284009 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst): first pass time = 0.70's, 192.579'MB/sec 2024/11/22-14:58:46.106751 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst): old prealloc_size = 147639500, real_size = 40358448 2024/11/22-14:58:46.106874 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.87 2024/11/22-14:58:46.106927 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:46.260764 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209487.sst): second pass time = 0.82's, 160.102'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2396.880'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.308'MB/sec write SST data time = 0.05's, 855.944'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44400.006'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.102'MB/sec zip pipeline throughput = 163.546'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.87 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2820 dict = 1.00 all = 3.2166 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.316875 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015532 GB avg = 0.012 KB total ukey num = 0.001294295 Billion total entry num = 0.001294295 Billion write speed all = 0.061081345 MB/sec (with seq num) write speed all = 0.060610332 MB/sec (without seq num) 2024/11/22-14:58:46.301387 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209487: 129430 keys, 41726266 bytes, temperature: kWarm 2024/11/22-14:58:46.301448 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258726301416, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209487, "file_size": 41726266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309968619, "table_properties": {"data_size": 40107104, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630103, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258724, "oldest_key_time": 0, "file_creation_time": 1732258724, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209487, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:46.338560 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst) /dev/shm/ToplingTemp/Topling-159797-0yz5di 2024/11/22-14:58:46.970929 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst): first pass time = 0.63's, 212.260'MB/sec 2024/11/22-14:58:47.770567 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst): old prealloc_size = 147639500, real_size = 40354896 2024/11/22-14:58:47.770681 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/22-14:58:47.770731 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:47.925691 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209488.sst): second pass time = 0.80's, 164.736'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2339.177'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.304'MB/sec write SST data time = 0.05's, 859.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41731.167'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.736'MB/sec zip pipeline throughput = 164.736'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.448505 GB avg = 1.017 KB (by entry num) total key len = 0.028475 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423726 Billion total entry num = 0.001423726 Billion write speed all = 0.067181764 MB/sec (with seq num) write speed all = 0.066663689 MB/sec (without seq num) 2024/11/22-14:58:47.962716 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209488: 129431 keys, 41722310 bytes, temperature: kWarm 2024/11/22-14:58:47.962773 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258727962742, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209488, "file_size": 41722310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309477986, "table_properties": {"data_size": 40103552, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629631, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258726, "oldest_key_time": 0, "file_creation_time": 1732258726, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209488, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:48.007606 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst) /dev/shm/ToplingTemp/Topling-159797-yzq5Yi 2024/11/22-14:58:48.446404 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst): first pass time = 0.44's, 305.905'MB/sec 2024/11/22-14:58:49.296231 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst): old prealloc_size = 147639500, real_size = 40358240 2024/11/22-14:58:49.296351 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.89 2024/11/22-14:58:49.296408 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:49.466754 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209489.sst): second pass time = 0.85's, 155.116'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 244.742'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2309.136'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.07's, 635.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39717.447'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.116'MB/sec zip pipeline throughput = 155.116'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2820 dict = 1.00 all = 3.2166 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.580135 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553151 Billion total entry num = 0.001553151 Billion write speed all = 0.073281660 MB/sec (with seq num) write speed all = 0.072716528 MB/sec (without seq num) 2024/11/22-14:58:49.506668 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209489: 129425 keys, 41726361 bytes, temperature: kWarm 2024/11/22-14:58:49.506748 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258729506704, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209489, "file_size": 41726361, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309843883, "table_properties": {"data_size": 40106896, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629506, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258727, "oldest_key_time": 0, "file_creation_time": 1732258727, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209489, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:49.557247 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst) /dev/shm/ToplingTemp/Topling-159797-djoq3g 2024/11/22-14:58:50.418545 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst): first pass time = 0.86's, 155.842'MB/sec 2024/11/22-14:58:51.641868 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst): old prealloc_size = 147639500, real_size = 40355104 2024/11/22-14:58:51.641981 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/22-14:58:51.642043 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:51.796224 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209490.sst): second pass time = 1.22's, 107.661'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.335'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2481.751'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.310'MB/sec write SST data time = 0.05's, 835.392'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 28044.119'MB/sec zip my value time = 1.22's, unzip length = 0.132'GB zip my value throughput = 107.661'MB/sec zip pipeline throughput = 124.004'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.711765 GB avg = 1.017 KB (by entry num) total key len = 0.033652 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682582 Billion total entry num = 0.001682582 Billion write speed all = 0.079377882 MB/sec (with seq num) write speed all = 0.078765720 MB/sec (without seq num) 2024/11/22-14:58:51.834917 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209490: 129431 keys, 41722919 bytes, temperature: kWarm 2024/11/22-14:58:51.834981 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258731834948, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209490, "file_size": 41722919, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310052168, "table_properties": {"data_size": 40103760, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629998, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258729, "oldest_key_time": 0, "file_creation_time": 1732258729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209490, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:51.871308 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst) /dev/shm/ToplingTemp/Topling-159797-iXQDRg 2024/11/22-14:58:52.379147 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst): first pass time = 0.51's, 264.313'MB/sec 2024/11/22-14:58:53.193508 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst): old prealloc_size = 147639500, real_size = 40350384 2024/11/22-14:58:53.193627 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.81 2024/11/22-14:58:53.193678 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:53.350076 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209491.sst): second pass time = 0.81's, 161.746'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.484'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2305.729'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 806.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 40087.763'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.746'MB/sec zip pipeline throughput = 161.746'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.81 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2826 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.843395 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001812015 Billion total entry num = 0.001812015 Billion write speed all = 0.085475819 MB/sec (with seq num) write speed all = 0.084816613 MB/sec (without seq num) 2024/11/22-14:58:53.388554 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209491: 129433 keys, 41718697 bytes, temperature: kWarm 2024/11/22-14:58:53.388616 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258733388585, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209491, "file_size": 41718697, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309916037, "table_properties": {"data_size": 40099040, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629746, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258731, "oldest_key_time": 0, "file_creation_time": 1732258731, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209491, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:53.428585 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst) /dev/shm/ToplingTemp/Topling-159797-WL4eTg 2024/11/22-14:58:53.937318 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst): first pass time = 0.51's, 263.853'MB/sec 2024/11/22-14:58:55.762616 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst): old prealloc_size = 147639500, real_size = 40358432 2024/11/22-14:58:55.762739 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.88 2024/11/22-14:58:55.762808 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:55.921846 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209492.sst): second pass time = 1.82's, 72.136'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2282.281'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 774.790'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 27482.261'MB/sec zip my value time = 1.82's, unzip length = 0.132'GB zip my value throughput = 72.136'MB/sec zip pipeline throughput = 116.238'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2166 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.975025 GB avg = 1.017 KB (by entry num) total key len = 0.038829 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941442 Billion total entry num = 0.001941442 Billion write speed all = 0.091568660 MB/sec (with seq num) write speed all = 0.090862451 MB/sec (without seq num) 2024/11/22-14:58:55.960228 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209492: 129427 keys, 41726257 bytes, temperature: kWarm 2024/11/22-14:58:55.960315 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258735960268, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209492, "file_size": 41726257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309974399, "table_properties": {"data_size": 40107088, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629958, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258733, "oldest_key_time": 0, "file_creation_time": 1732258733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209492, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:56.009952 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst) /dev/shm/ToplingTemp/Topling-159797-H5Rn2j 2024/11/22-14:58:56.621628 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst): first pass time = 0.61's, 219.448'MB/sec 2024/11/22-14:58:57.449849 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst): old prealloc_size = 147639500, real_size = 40355504 2024/11/22-14:58:57.449963 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.85 2024/11/22-14:58:57.450017 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:58:57.604222 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209493.sst): second pass time = 0.83's, 159.215'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2383.694'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.312'MB/sec write SST data time = 0.05's, 823.474'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43255.023'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.215'MB/sec zip pipeline throughput = 159.215'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 21 bytes = 64 } seq expand size = 147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.106655 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070872 Billion total entry num = 0.002070872 Billion write speed all = 0.097664017 MB/sec (with seq num) write speed all = 0.096910784 MB/sec (without seq num) 2024/11/22-14:58:57.638976 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209493: 129430 keys, 41723211 bytes, temperature: kWarm 2024/11/22-14:58:57.639052 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258737639010, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209493, "file_size": 41723211, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309981722, "table_properties": {"data_size": 40104160, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629675, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258735, "oldest_key_time": 0, "file_creation_time": 1732258735, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209493, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:58:57.679921 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst) /dev/shm/ToplingTemp/Topling-159797-5G8Bni 2024/11/22-14:58:58.186734 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst): first pass time = 0.51's, 264.850'MB/sec 2024/11/22-14:59:00.231236 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst): old prealloc_size = 147639500, real_size = 40360160 2024/11/22-14:59:00.231377 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.90 2024/11/22-14:59:00.231456 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:59:00.461988 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209494.sst): second pass time = 2.04's, 64.405'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 180.851'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1967.982'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.195'MB/sec write SST data time = 0.06's, 644.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36123.345'MB/sec zip my value time = 2.04's, unzip length = 0.132'GB zip my value throughput = 64.405'MB/sec zip pipeline throughput = 114.924'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.90 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.90 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 15 bytes = 48 } seq expand size = 105 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2818 dict = 1.00 all = 3.2165 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.238284 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026404 GB avg = 0.012 KB total ukey num = 0.002200297 Billion total entry num = 0.002200297 Billion write speed all = 0.103752882 MB/sec (with seq num) write speed all = 0.102952678 MB/sec (without seq num) 2024/11/22-14:59:00.506956 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209494: 129425 keys, 41727812 bytes, temperature: kWarm 2024/11/22-14:59:00.507048 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258740506993, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209494, "file_size": 41727812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309949864, "table_properties": {"data_size": 40108816, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629481, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258737, "oldest_key_time": 0, "file_creation_time": 1732258737, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209494, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:59:00.553278 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst) /dev/shm/ToplingTemp/Topling-159797-lyPgbk 2024/11/22-14:59:01.010916 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst): first pass time = 0.46's, 293.317'MB/sec 2024/11/22-14:59:02.103068 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst): old prealloc_size = 147639500, real_size = 40348800 2024/11/22-14:59:02.103196 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129436 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.94 avg-zval = 309.79 2024/11/22-14:59:02.103249 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:59:02.262590 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209495.sst): second pass time = 1.09's, 120.590'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.531'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2247.679'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.294'MB/sec write SST data time = 0.05's, 843.883'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35873.175'MB/sec zip my value time = 1.09's, unzip length = 0.132'GB zip my value throughput = 120.590'MB/sec zip pipeline throughput = 146.713'MB/sec entries = 129436 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.94 avg-zval = 309.79 usrkeys = 129436 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2827 dict = 1.00 all = 3.2174 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.369914 GB avg = 1.017 KB (by entry num) total key len = 0.046595 GB avg = 0.020 KB total ukey len = 0.027957 GB avg = 0.012 KB total ukey num = 0.002329733 Billion total entry num = 0.002329733 Billion write speed all = 0.109845442 MB/sec (with seq num) write speed all = 0.108998234 MB/sec (without seq num) 2024/11/22-14:59:02.301909 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209495: 129436 keys, 41716914 bytes, temperature: kWarm 2024/11/22-14:59:02.301963 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258742301935, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209495, "file_size": 41716914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310033125, "table_properties": {"data_size": 40097456, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588720, "raw_average_key_size": 20, "raw_value_size": 131629258, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258740, "oldest_key_time": 0, "file_creation_time": 1732258740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209495, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:59:02.339578 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst) /dev/shm/ToplingTemp/Topling-159797-M8Xplh 2024/11/22-14:59:02.828615 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst): first pass time = 0.49's, 274.477'MB/sec 2024/11/22-14:59:03.634918 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst): old prealloc_size = 147639500, real_size = 40354688 2024/11/22-14:59:03.635054 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.84 2024/11/22-14:59:03.635109 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:59:03.790753 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209496.sst): second pass time = 0.80's, 163.567'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2266.161'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 815.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43849.309'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 163.567'MB/sec zip pipeline throughput = 163.567'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.84 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.501544 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459165 Billion total entry num = 0.002459165 Billion write speed all = 0.115938475 MB/sec (with seq num) write speed all = 0.115044261 MB/sec (without seq num) 2024/11/22-14:59:03.829081 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209496: 129432 keys, 41722280 bytes, temperature: kWarm 2024/11/22-14:59:03.829148 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258743829113, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209496, "file_size": 41722280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309993847, "table_properties": {"data_size": 40103344, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131630046, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258742, "oldest_key_time": 0, "file_creation_time": 1732258742, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209496, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:59:03.866077 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst) /dev/shm/ToplingTemp/Topling-159797-SKtDei 2024/11/22-14:59:04.478807 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst): first pass time = 0.61's, 219.067'MB/sec 2024/11/22-14:59:05.294622 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst): old prealloc_size = 147639500, real_size = 40354880 2024/11/22-14:59:05.294746 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.85 2024/11/22-14:59:05.294811 140025037178432 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:59:05.448548 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209497.sst): second pass time = 0.82's, 161.473'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.074'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2294.487'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.310'MB/sec write SST data time = 0.05's, 841.791'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37405.233'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.473'MB/sec zip pipeline throughput = 161.473'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.633173 GB avg = 1.017 KB (by entry num) total key len = 0.051772 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588592 Billion total entry num = 0.002588592 Billion write speed all = 0.122029919 MB/sec (with seq num) write speed all = 0.121088714 MB/sec (without seq num) 2024/11/22-14:59:05.484949 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209497: 129427 keys, 41722760 bytes, temperature: kWarm 2024/11/22-14:59:05.485015 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258745484982, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209497, "file_size": 41722760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310022784, "table_properties": {"data_size": 40103536, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629704, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258743, "oldest_key_time": 0, "file_creation_time": 1732258743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209497, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:59:05.523360 140025037178432 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209498.sst) /dev/shm/ToplingTemp/Topling-159797-lE1hjh 2024/11/22-14:59:05.735046 140025037178432 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209498.sst): first pass time = 0.21's, 285.092'MB/sec 2024/11/22-14:59:06.171335 140025037178432 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209498.sst): old prealloc_size = 147639500, real_size = 18134920 2024/11/22-14:59:06.171389 140025037178432 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209498.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 58181 prefix = 4 raw-key = 0.0005 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0592 GB zip-val = 0.0181 GB avg-val =1017.07 avg-zval = 311.44 2024/11/22-14:59:06.240913 140025037178432 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00/209498.sst): second pass time = 0.44's, 135.817'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.07's, 269.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.482'MB/sec (index lex order gen) rebuild zvType time = 0.05's, 0.300'MB/sec write SST data time = 0.02's, 888.310'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.599'MB, throughput = 13497.037'MB/sec zip my value time = 0.44's, unzip length = 0.059'GB zip my value throughput = 135.817'MB/sec zip pipeline throughput = 135.817'MB/sec entries = 58181 avg-key = 20.00 avg-zkey = 0.00 avg-val = 1017.07 avg-zval = 311.44 usrkeys = 58181 avg-key = 12.00 avg-zkey = 0.00 avg-val = 1017.07 avg-zval = 311.44 TagRS{ kinds = RS_Key0_TagN bytes = 9200 } TagArray{ size = 5 bytes = 32 } seq expand size = 35 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0007 GB value = 0.0592 GB dict = 0.60 MB all = 0.0603 GB } __ZipSize{ index = 0.0000 GB value = 0.0181 GB dict = 0.60 MB all = 0.0187 GB } UnZip/Zip{ index =3636.3125 value = 3.2657 dict = 1.00 all = 3.2215 } Zip/UnZip{ index = 0.0003 value = 0.3062 dict = 1.00 all = 0.3104 } ---------------------------- total value len = 2.692348 GB avg = 1.017 KB (by entry num) total key len = 0.052935 GB avg = 0.020 KB total ukey len = 0.031761 GB avg = 0.012 KB total ukey num = 0.002646773 Billion total entry num = 0.002646773 Billion write speed all = 0.124767759 MB/sec (with seq num) write speed all = 0.123805434 MB/sec (without seq num) 2024/11/22-14:59:06.267975 140025037178432 [compaction_job.cc:1948] [default] [JOB 1824] Generated table #209498: 58181 keys, 18729639 bytes, temperature: kWarm 2024/11/22-14:59:06.268051 140025037178432 EVENT_LOG_v1 {"time_micros": 1732258746268007, "cf_name": "default", "job": 1824, "event": "table_file_creation", "file_number": 209498, "file_size": 18729639, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3309916516, "table_properties": {"data_size": 18120016, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1163620, "raw_average_key_size": 20, "raw_value_size": 59174119, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 58181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732258745, "oldest_key_time": 0, "file_creation_time": 1732258745, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209498, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:59:06.284345 140025037178432 [compaction_job.cc:703] job-01824: subcompact[0], size: 0.854327 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01824/att-00 [209478,209479,209480,209481,209482,209483,209484,209485,209486,209487,209488,209489,209490,209491,209492,209493,209494,209495,209496,209497,209498] 2024/11/22-14:59:06.439894 140025037178432 (Original Log Time 2024/11/22-14:58:28.395492) EVENT_LOG_v1 {"time_micros": 1732258708395440, "job": 1824, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [209473], "files_L3": [209401, 209402, 209403, 209422, 209426, 209424, 209425, 209423, 209427, 209433, 209434, 209435, 209436, 209437, 209438, 209439, 209440, 209441, 209442, 209444, 209443], "score": 0.895293, "input_data_size": 867109793, "oldest_snapshot_seqno": 3310124106} 2024/11/22-14:59:06.439899 140025037178432 (Original Log Time 2024/11/22-14:59:06.438863) [compaction_job.cc:2022] [default] [JOB 1824] Compacted 1@2 + 21@3 files to L3 => 854326670 bytes 2024/11/22-14:59:06.439902 140025037178432 (Original Log Time 2024/11/22-14:59:06.439818) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.9 rd, 22.5 wr, level 3, files in(1, 21) out(21 +0 blob) MB in(12.6, 814.3 +0.0 blob) out(814.7 +0.0 blob), read-write-amplify(130.0) write-amplify(64.5) OK, records in: 2659458, records dropped: 12685 output_compression: Snappy 2024/11/22-14:59:06.439907 140025037178432 (Original Log Time 2024/11/22-14:59:06.439867) EVENT_LOG_v1 {"time_micros": 1732258746439843, "cf": "default", "job": 1824, "event": "compaction_finished", "compaction_time_micros": 37888863, "compaction_time_cpu_micros": 7676371, "output_level": 3, "num_output_files": 21, "total_output_size": 854326670, "num_input_records": 2659458, "num_output_records": 2646773, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 561164090, "file_range_sync_nanos": 0, "file_fsync_nanos": 19682, "file_prepare_write_nanos": 43211, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/22-14:59:06.572352 140025037178432 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732258708,"executesMs":38183,"compactionJobId":1824,"attempt":0,"compactionInputRawBytes":2757859078,"compactionInputZipBytes":867109793,"compactionOutputRawBytes":0,"compactionOutputZipBytes":854326670,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e3d83c53a0f23541e92ec3f2602cd4c2-685c6a76e05bacf7-00"}}, response = 2024/11/22-14:59:06.572421 140025037178432 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01824/att-00: olev 3, work 38.183 s, result 6.612 ms, install 1.066 ms, input{raw 2.568 GiB zip 826.940 MiB}