2024/11/23-21:58:55.337800 139957601228352 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00: fork to child time = 0.007749 sec 2024/11/23-21:58:55.338580 139957601228352 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00 2024/11/23-21:58:55.346186 139957601228352 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00 2024/11/23-21:58:55.346774 139957601228352 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:58:55.347033 139957601228352 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:58:55.401536 139957601228352 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36608/att-00/MANIFEST-206347 2024/11/23-21:58:55.441439 139957601228352 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36608/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 282341, last_sequence is 4286761365, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 282236 2024/11/23-21:58:55.441445 139957601228352 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:58:55.441797 139957601228352 [version_set.cc:5721] Creating manifest 282341 2024/11/23-21:58:55.479359 139957601228352 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00: bottommost_level: fake = 1, rpc = 1 2024/11/23-21:58:55.498215 139957601228352 [compaction_job.cc:2437] [default] [JOB 36608] Compacting 53@4 files to L5, score -1.00, subcompactions 1 : 1 2024/11/23-21:58:55.498243 139957601228352 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 27409 Base level 4, inputs: [281393(79MB) 281394(79MB) 281395(79MB) 281396(79MB) 281397(79MB) 281398(79MB) 281399(79MB) 281400(79MB) 281401(79MB) 281402(79MB) 281403(79MB) 281404(79MB) 281405(79MB) 281406(79MB) 281407(79MB) 281408(79MB) 281409(79MB) 281410(79MB) 281411(79MB) 281412(29MB) 281606(79MB) 281607(79MB) 281608(79MB) 281609(79MB) 281610(79MB) 281611(79MB) 281612(79MB) 281613(79MB) 281614(79MB) 281615(79MB) 281616(79MB) 281617(79MB) 281618(79MB) 281619(79MB) 281620(79MB) 281621(79MB) 281622(79MB) 281623(79MB) 281624(79MB) 281625(79MB) 281626(79MB) 281627(79MB) 281628(79MB) 281629(79MB) 281630(79MB) 281631(79MB) 281632(79MB) 281633(79MB) 281634(79MB) 281635(79MB) 281636(79MB) 281637(79MB) 281638(79MB)] 2024/11/23-21:58:55.503893 139957601228352 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:58:55.503924 139957601228352 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:58:55.503945 139957601228352 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:58:55.503951 139957601228352 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:58:55.503961 139957601228352 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:58:56.573356 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst) /dev/shm/ToplingTemp/Topling-199889-BhLFLT 2024/11/23-21:58:58.278494 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst): first pass time = 1.71's, 314.862'MB/sec 2024/11/23-21:59:05.875764 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst): old prealloc_size = 590558003, real_size = 165151344 2024/11/23-21:59:05.876129 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517718 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-21:59:05.876198 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:06.559936 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282342.sst): second pass time = 7.59's, 69.331'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 248.908'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2571.456'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.263'MB/sec write SST data time = 0.19's, 888.538'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 140266.128'MB/sec zip my value time = 7.59's, unzip length = 0.527'GB zip my value throughput = 69.331'MB/sec zip pipeline throughput = 120.439'MB/sec entries = 517718 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517718 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 9.375136 GB avg = 0.177 KB (by entry num) total key len = 1.451175 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068153 Billion total entry num = 0.052875882 Billion write speed all = 0.230178858 MB/sec (with seq num) write speed all = 0.221185280 MB/sec (without seq num) 2024/11/23-21:59:06.752180 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282342: 517718 keys, 170297884 bytes, temperature: kWarm 2024/11/23-21:59:06.752262 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370346752208, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282342, "file_size": 170297884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164151200, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354360, "raw_average_key_size": 20, "raw_value_size": 526516637, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370336, "oldest_key_time": 0, "file_creation_time": 1732370336, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282342, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:06.852429 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst) /dev/shm/ToplingTemp/Topling-199889-F1UaLR 2024/11/23-21:59:09.438587 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst): first pass time = 2.59's, 207.597'MB/sec 2024/11/23-21:59:15.585527 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst): old prealloc_size = 590558003, real_size = 165144000 2024/11/23-21:59:15.585924 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517725 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1641 GB avg-val =1016.98 avg-zval = 317.05 2024/11/23-21:59:15.585988 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:16.304474 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282343.sst): second pass time = 6.14's, 85.693'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 236.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2439.390'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.250'MB/sec write SST data time = 0.20's, 847.361'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 181564.028'MB/sec zip my value time = 6.14's, unzip length = 0.527'GB zip my value throughput = 85.693'MB/sec zip pipeline throughput = 121.383'MB/sec entries = 517725 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 317.05 usrkeys = 517725 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.98 avg-zval = 317.05 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1641 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1367 value = 3.2077 dict = 1.00 all = 3.1527 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 9.901653 GB avg = 0.185 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585878 Billion total entry num = 0.053393607 Billion write speed all = 0.241543264 MB/sec (with seq num) write speed all = 0.232463508 MB/sec (without seq num) 2024/11/23-21:59:16.537881 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282343: 517725 keys, 170289310 bytes, temperature: kWarm 2024/11/23-21:59:16.537943 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370356537912, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282343, "file_size": 170289310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164143856, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354500, "raw_average_key_size": 20, "raw_value_size": 526516629, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517725, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370346, "oldest_key_time": 0, "file_creation_time": 1732370346, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:16.638302 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst) /dev/shm/ToplingTemp/Topling-199889-x26nfT 2024/11/23-21:59:19.125144 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst): first pass time = 2.49's, 215.888'MB/sec 2024/11/23-21:59:24.012603 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst): old prealloc_size = 590558003, real_size = 165155760 2024/11/23-21:59:24.012904 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517718 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.08 2024/11/23-21:59:24.012942 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:24.828427 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282344.sst): second pass time = 4.89's, 107.773'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 208.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.526'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.203'MB/sec write SST data time = 0.18's, 961.993'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 127477.865'MB/sec zip my value time = 4.89's, unzip length = 0.527'GB zip my value throughput = 107.773'MB/sec zip pipeline throughput = 113.642'MB/sec entries = 517718 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 usrkeys = 517718 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 10.428170 GB avg = 0.193 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103596 Billion total entry num = 0.053911325 Billion write speed all = 0.252909526 MB/sec (with seq num) write speed all = 0.243743392 MB/sec (without seq num) 2024/11/23-21:59:25.020406 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282344: 517718 keys, 170301913 bytes, temperature: kWarm 2024/11/23-21:59:25.020476 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370365020441, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282344, "file_size": 170301913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155616, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354360, "raw_average_key_size": 20, "raw_value_size": 526516904, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370356, "oldest_key_time": 0, "file_creation_time": 1732370356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:25.114149 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst) /dev/shm/ToplingTemp/Topling-199889-SrsMvS 2024/11/23-21:59:27.683185 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst): first pass time = 2.57's, 208.981'MB/sec 2024/11/23-21:59:32.537427 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst): old prealloc_size = 590558003, real_size = 165172736 2024/11/23-21:59:32.537746 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517706 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.02 avg-zval = 317.12 2024/11/23-21:59:32.537783 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:33.180574 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282345.sst): second pass time = 4.85's, 108.516'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 264.830'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3051.208'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.278'MB/sec write SST data time = 0.18's, 963.467'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 170259.631'MB/sec zip my value time = 4.85's, unzip length = 0.527'GB zip my value throughput = 108.516'MB/sec zip pipeline throughput = 108.516'MB/sec entries = 517706 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.12 usrkeys = 517706 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1364 value = 3.2071 dict = 1.00 all = 3.1522 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 10.954688 GB avg = 0.201 KB (by entry num) total key len = 1.482238 GB avg = 0.027 KB total ukey len = 1.022728 GB avg = 0.015 KB total ukey num = 0.067621302 Billion total entry num = 0.054429031 Billion write speed all = 0.264272645 MB/sec (with seq num) write speed all = 0.255020131 MB/sec (without seq num) 2024/11/23-21:59:33.366036 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282345: 517706 keys, 170318946 bytes, temperature: kWarm 2024/11/23-21:59:33.366103 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370373366070, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282345, "file_size": 170318946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164172592, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354120, "raw_average_key_size": 20, "raw_value_size": 526517540, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517706, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370365, "oldest_key_time": 0, "file_creation_time": 1732370365, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:33.460709 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst) /dev/shm/ToplingTemp/Topling-199889-BS65zS 2024/11/23-21:59:36.087903 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst): first pass time = 2.63's, 204.355'MB/sec 2024/11/23-21:59:41.098908 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst): old prealloc_size = 590558003, real_size = 165152512 2024/11/23-21:59:41.099253 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517715 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-21:59:41.099293 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:41.738529 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282346.sst): second pass time = 5.01's, 105.117'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 266.260'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2892.437'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.280'MB/sec write SST data time = 0.18's, 963.187'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 162640.253'MB/sec zip my value time = 5.01's, unzip length = 0.527'GB zip my value throughput = 105.117'MB/sec zip pipeline throughput = 105.117'MB/sec entries = 517715 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517715 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 11.481205 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139017 Billion total entry num = 0.054946746 Billion write speed all = 0.275630528 MB/sec (with seq num) write speed all = 0.266291704 MB/sec (without seq num) 2024/11/23-21:59:41.927289 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282346: 517715 keys, 170299089 bytes, temperature: kWarm 2024/11/23-21:59:41.927357 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370381927324, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282346, "file_size": 170299089, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164152368, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354300, "raw_average_key_size": 20, "raw_value_size": 526517485, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517715, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370373, "oldest_key_time": 0, "file_creation_time": 1732370373, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:42.036455 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst) /dev/shm/ToplingTemp/Topling-199889-p8mcUT 2024/11/23-21:59:44.710687 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst): first pass time = 2.67's, 200.760'MB/sec 2024/11/23-21:59:49.287480 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst): old prealloc_size = 590558003, real_size = 165147936 2024/11/23-21:59:49.287793 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517721 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1641 GB avg-val =1016.99 avg-zval = 317.06 2024/11/23-21:59:49.287834 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:49.932202 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282347.sst): second pass time = 4.57's, 115.094'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 264.145'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3193.321'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.278'MB/sec write SST data time = 0.18's, 951.926'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 157581.936'MB/sec zip my value time = 4.57's, unzip length = 0.527'GB zip my value throughput = 115.094'MB/sec zip pipeline throughput = 115.094'MB/sec entries = 517721 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.06 usrkeys = 517721 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.06 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1641 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2076 dict = 1.00 all = 3.1526 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 12.007722 GB avg = 0.216 KB (by entry num) total key len = 1.502947 GB avg = 0.027 KB total ukey len = 1.035154 GB avg = 0.015 KB total ukey num = 0.068656738 Billion total entry num = 0.055464467 Billion write speed all = 0.286986496 MB/sec (with seq num) write speed all = 0.277561321 MB/sec (without seq num) 2024/11/23-21:59:50.140243 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282347: 517721 keys, 170294218 bytes, temperature: kWarm 2024/11/23-21:59:50.140314 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370390140278, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282347, "file_size": 170294218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164147792, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354420, "raw_average_key_size": 20, "raw_value_size": 526517092, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370382, "oldest_key_time": 0, "file_creation_time": 1732370382, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:50.236249 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst) /dev/shm/ToplingTemp/Topling-199889-ZKBBcU 2024/11/23-21:59:53.000923 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst): first pass time = 2.76's, 194.193'MB/sec 2024/11/23-21:59:57.666714 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst): old prealloc_size = 590558003, real_size = 165153072 2024/11/23-21:59:57.667055 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517723 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.07 2024/11/23-21:59:57.667103 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-21:59:58.316126 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282348.sst): second pass time = 4.66's, 112.902'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2907.303'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 948.752'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 151281.234'MB/sec zip my value time = 4.66's, unzip length = 0.527'GB zip my value throughput = 112.902'MB/sec zip pipeline throughput = 112.902'MB/sec entries = 517723 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 usrkeys = 517723 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1367 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 12.534239 GB avg = 0.224 KB (by entry num) total key len = 1.513302 GB avg = 0.027 KB total ukey len = 1.041366 GB avg = 0.015 KB total ukey num = 0.069174461 Billion total entry num = 0.055982190 Billion write speed all = 0.298337307 MB/sec (with seq num) write speed all = 0.288825848 MB/sec (without seq num) 2024/11/23-21:59:58.505109 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282348: 517723 keys, 170299019 bytes, temperature: kWarm 2024/11/23-21:59:58.505187 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370398505150, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282348, "file_size": 170299019, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164152928, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354460, "raw_average_key_size": 20, "raw_value_size": 526517033, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517723, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370390, "oldest_key_time": 0, "file_creation_time": 1732370390, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:59:58.605710 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst) /dev/shm/ToplingTemp/Topling-199889-rZAMZT 2024/11/23-22:00:01.413552 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst): first pass time = 2.81's, 191.207'MB/sec 2024/11/23-22:00:06.051400 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst): old prealloc_size = 590558003, real_size = 165838208 2024/11/23-22:00:06.051704 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517713 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1648 GB avg-val =1017.01 avg-zval = 318.40 2024/11/23-22:00:06.051747 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:06.700399 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282349.sst): second pass time = 4.63's, 113.612'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 263.466'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3304.414'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 948.589'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 159995.660'MB/sec zip my value time = 4.63's, unzip length = 0.527'GB zip my value throughput = 113.612'MB/sec zip pipeline throughput = 113.612'MB/sec entries = 517713 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 318.40 usrkeys = 517713 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 318.40 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1648 GB dict = 5.27 MB all = 0.1710 GB } UnZip/Zip{ index = 7.1365 value = 3.1941 dict = 1.00 all = 3.1399 } Zip/UnZip{ index = 0.1401 value = 0.3131 dict = 1.00 all = 0.3185 } ---------------------------- total value len = 13.060756 GB avg = 0.231 KB (by entry num) total key len = 1.523656 GB avg = 0.027 KB total ukey len = 1.047579 GB avg = 0.015 KB total ukey num = 0.069692174 Billion total entry num = 0.056499903 Billion write speed all = 0.309684072 MB/sec (with seq num) write speed all = 0.300086362 MB/sec (without seq num) 2024/11/23-22:00:06.887582 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282349: 517713 keys, 170983525 bytes, temperature: kWarm 2024/11/23-22:00:06.887654 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370406887617, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282349, "file_size": 170983525, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164838064, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354260, "raw_average_key_size": 20, "raw_value_size": 526517163, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517713, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370398, "oldest_key_time": 0, "file_creation_time": 1732370398, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:06.980437 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst) /dev/shm/ToplingTemp/Topling-199889-CqDQmQ 2024/11/23-22:00:09.651894 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst): first pass time = 2.67's, 200.969'MB/sec 2024/11/23-22:00:14.241477 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst): old prealloc_size = 590558003, real_size = 165149744 2024/11/23-22:00:14.241797 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517719 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1641 GB avg-val =1016.99 avg-zval = 317.06 2024/11/23-22:00:14.241839 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:14.891102 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282350.sst): second pass time = 4.59's, 114.781'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.165'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.867'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 943.898'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 162052.923'MB/sec zip my value time = 4.59's, unzip length = 0.527'GB zip my value throughput = 114.781'MB/sec zip pipeline throughput = 114.781'MB/sec entries = 517719 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.06 usrkeys = 517719 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.06 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1641 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1526 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 13.587273 GB avg = 0.238 KB (by entry num) total key len = 1.534010 GB avg = 0.027 KB total ukey len = 1.053792 GB avg = 0.015 KB total ukey num = 0.070209893 Billion total entry num = 0.057017622 Billion write speed all = 0.321028110 MB/sec (with seq num) write speed all = 0.311344138 MB/sec (without seq num) 2024/11/23-22:00:15.081944 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282350: 517719 keys, 170296418 bytes, temperature: kWarm 2024/11/23-22:00:15.082027 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370415081989, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282350, "file_size": 170296418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164149600, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354380, "raw_average_key_size": 20, "raw_value_size": 526516667, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370406, "oldest_key_time": 0, "file_creation_time": 1732370406, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:15.174886 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst) /dev/shm/ToplingTemp/Topling-199889-pbo4IS 2024/11/23-22:00:18.285062 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst): first pass time = 3.11's, 172.620'MB/sec 2024/11/23-22:00:22.877360 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst): old prealloc_size = 590558003, real_size = 165575744 2024/11/23-22:00:22.877696 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517721 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1646 GB avg-val =1016.99 avg-zval = 317.88 2024/11/23-22:00:22.877735 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:23.527343 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282351.sst): second pass time = 4.59's, 114.703'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.666'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2992.523'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 956.983'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 110231.988'MB/sec zip my value time = 4.59's, unzip length = 0.527'GB zip my value throughput = 114.703'MB/sec zip pipeline throughput = 114.703'MB/sec entries = 517721 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.88 usrkeys = 517721 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.88 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1646 GB dict = 5.27 MB all = 0.1707 GB } UnZip/Zip{ index = 7.1366 value = 3.1992 dict = 1.00 all = 3.1447 } Zip/UnZip{ index = 0.1401 value = 0.3126 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 14.113790 GB avg = 0.245 KB (by entry num) total key len = 1.544365 GB avg = 0.027 KB total ukey len = 1.060004 GB avg = 0.015 KB total ukey num = 0.070727614 Billion total entry num = 0.057535343 Billion write speed all = 0.332365068 MB/sec (with seq num) write speed all = 0.322594957 MB/sec (without seq num) 2024/11/23-22:00:23.711546 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282351: 517721 keys, 170721920 bytes, temperature: kWarm 2024/11/23-22:00:23.711617 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370423711581, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282351, "file_size": 170721920, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164575600, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354420, "raw_average_key_size": 20, "raw_value_size": 526517045, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370415, "oldest_key_time": 0, "file_creation_time": 1732370415, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:23.813520 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst) /dev/shm/ToplingTemp/Topling-199889-FzJR3P 2024/11/23-22:00:26.466880 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst): first pass time = 2.65's, 202.340'MB/sec 2024/11/23-22:00:31.059075 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst): old prealloc_size = 590558003, real_size = 165773856 2024/11/23-22:00:31.059414 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517715 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1648 GB avg-val =1017.00 avg-zval = 318.27 2024/11/23-22:00:31.059455 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:31.709132 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282352.sst): second pass time = 4.59's, 114.714'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.938'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2924.866'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.273'MB/sec write SST data time = 0.18's, 973.195'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 161475.464'MB/sec zip my value time = 4.59's, unzip length = 0.527'GB zip my value throughput = 114.714'MB/sec zip pipeline throughput = 114.714'MB/sec entries = 517715 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 318.27 usrkeys = 517715 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 318.27 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1648 GB dict = 5.27 MB all = 0.1709 GB } UnZip/Zip{ index = 7.1366 value = 3.1954 dict = 1.00 all = 3.1411 } Zip/UnZip{ index = 0.1401 value = 0.3130 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 14.640307 GB avg = 0.252 KB (by entry num) total key len = 1.554719 GB avg = 0.027 KB total ukey len = 1.066217 GB avg = 0.015 KB total ukey num = 0.071245329 Billion total entry num = 0.058053058 Billion write speed all = 0.343701176 MB/sec (with seq num) write speed all = 0.333844864 MB/sec (without seq num) 2024/11/23-22:00:31.926288 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282352: 517715 keys, 170919132 bytes, temperature: kWarm 2024/11/23-22:00:31.926360 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370431926325, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282352, "file_size": 170919132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164773712, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354300, "raw_average_key_size": 20, "raw_value_size": 526516721, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517715, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370423, "oldest_key_time": 0, "file_creation_time": 1732370423, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:32.021638 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst) /dev/shm/ToplingTemp/Topling-199889-2C6CBT 2024/11/23-22:00:34.759514 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst): first pass time = 2.74's, 196.094'MB/sec 2024/11/23-22:00:39.373801 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst): old prealloc_size = 590558003, real_size = 165147712 2024/11/23-22:00:39.374201 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517714 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1641 GB avg-val =1017.00 avg-zval = 317.06 2024/11/23-22:00:39.374251 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:40.021708 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282353.sst): second pass time = 4.61's, 114.163'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.844'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2445.139'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.274'MB/sec write SST data time = 0.17's, 974.457'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 167084.110'MB/sec zip my value time = 4.61's, unzip length = 0.527'GB zip my value throughput = 114.163'MB/sec zip pipeline throughput = 114.163'MB/sec entries = 517714 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.06 usrkeys = 517714 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.06 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1641 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2076 dict = 1.00 all = 3.1526 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 15.166824 GB avg = 0.259 KB (by entry num) total key len = 1.565073 GB avg = 0.027 KB total ukey len = 1.072429 GB avg = 0.015 KB total ukey num = 0.071763043 Billion total entry num = 0.058570772 Billion write speed all = 0.355032369 MB/sec (with seq num) write speed all = 0.345089913 MB/sec (without seq num) 2024/11/23-22:00:40.213692 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282353: 517714 keys, 170293664 bytes, temperature: kWarm 2024/11/23-22:00:40.213764 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370440213727, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282353, "file_size": 170293664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164147568, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354280, "raw_average_key_size": 20, "raw_value_size": 526517060, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370432, "oldest_key_time": 0, "file_creation_time": 1732370432, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:40.304920 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst) /dev/shm/ToplingTemp/Topling-199889-jqgkPP 2024/11/23-22:00:42.822476 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst): first pass time = 2.52's, 213.255'MB/sec 2024/11/23-22:00:47.422795 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst): old prealloc_size = 590558003, real_size = 165164480 2024/11/23-22:00:47.423115 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517714 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.09 2024/11/23-22:00:47.423167 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:48.072604 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282354.sst): second pass time = 4.60's, 114.493'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.100'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3123.127'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 952.265'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.271'MB, throughput = 164600.506'MB/sec zip my value time = 4.60's, unzip length = 0.527'GB zip my value throughput = 114.493'MB/sec zip pipeline throughput = 114.493'MB/sec entries = 517714 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.09 usrkeys = 517714 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2073 dict = 1.00 all = 3.1523 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 15.693341 GB avg = 0.266 KB (by entry num) total key len = 1.575428 GB avg = 0.027 KB total ukey len = 1.078642 GB avg = 0.015 KB total ukey num = 0.072280757 Billion total entry num = 0.059088486 Billion write speed all = 0.366361598 MB/sec (with seq num) write speed all = 0.356332972 MB/sec (without seq num) 2024/11/23-22:00:48.261521 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282354: 517714 keys, 170309262 bytes, temperature: kWarm 2024/11/23-22:00:48.261589 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370448261553, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282354, "file_size": 170309262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164164336, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354280, "raw_average_key_size": 20, "raw_value_size": 526516932, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370440, "oldest_key_time": 0, "file_creation_time": 1732370440, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:48.357659 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst) /dev/shm/ToplingTemp/Topling-199889-85EAJP 2024/11/23-22:00:50.900794 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst): first pass time = 2.54's, 211.111'MB/sec 2024/11/23-22:00:55.514261 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst): old prealloc_size = 590558003, real_size = 165156752 2024/11/23-22:00:55.514564 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517719 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1016.99 avg-zval = 317.08 2024/11/23-22:00:55.514604 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:00:56.162633 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282355.sst): second pass time = 4.61's, 114.169'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3356.421'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 962.346'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 151684.407'MB/sec zip my value time = 4.61's, unzip length = 0.527'GB zip my value throughput = 114.169'MB/sec zip pipeline throughput = 114.169'MB/sec entries = 517719 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 usrkeys = 517719 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1016.99 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 16.219858 GB avg = 0.272 KB (by entry num) total key len = 1.585782 GB avg = 0.027 KB total ukey len = 1.084854 GB avg = 0.015 KB total ukey num = 0.072798476 Billion total entry num = 0.059606205 Billion write speed all = 0.377686656 MB/sec (with seq num) write speed all = 0.367571898 MB/sec (without seq num) 2024/11/23-22:00:56.349905 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282355: 517719 keys, 170303066 bytes, temperature: kWarm 2024/11/23-22:00:56.349998 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370456349940, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282355, "file_size": 170303066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164156608, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354380, "raw_average_key_size": 20, "raw_value_size": 526517476, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370448, "oldest_key_time": 0, "file_creation_time": 1732370448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:00:56.441605 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst) /dev/shm/ToplingTemp/Topling-199889-mnBl0T 2024/11/23-22:00:59.005385 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst): first pass time = 2.56's, 209.410'MB/sec 2024/11/23-22:01:03.782877 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst): old prealloc_size = 590558003, real_size = 165157600 2024/11/23-22:01:03.783255 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517707 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.02 avg-zval = 317.09 2024/11/23-22:01:03.783298 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:04.433637 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282356.sst): second pass time = 4.78's, 110.250'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.709'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2606.941'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 947.548'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 160891.001'MB/sec zip my value time = 4.78's, unzip length = 0.527'GB zip my value throughput = 110.250'MB/sec zip pipeline throughput = 110.250'MB/sec entries = 517707 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.09 usrkeys = 517707 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 16.746375 GB avg = 0.279 KB (by entry num) total key len = 1.596136 GB avg = 0.027 KB total ukey len = 1.091067 GB avg = 0.015 KB total ukey num = 0.073316183 Billion total entry num = 0.060123912 Billion write speed all = 0.389006319 MB/sec (with seq num) write speed all = 0.378805499 MB/sec (without seq num) 2024/11/23-22:01:04.619055 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282356: 517707 keys, 170303907 bytes, temperature: kWarm 2024/11/23-22:01:04.619135 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370464619099, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282356, "file_size": 170303907, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164157456, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354140, "raw_average_key_size": 20, "raw_value_size": 526516902, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370456, "oldest_key_time": 0, "file_creation_time": 1732370456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:04.718199 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst) /dev/shm/ToplingTemp/Topling-199889-iESSNQ 2024/11/23-22:01:07.297881 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst): first pass time = 2.58's, 208.120'MB/sec 2024/11/23-22:01:11.924283 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst): old prealloc_size = 590558003, real_size = 165157024 2024/11/23-22:01:11.924583 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517713 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.08 2024/11/23-22:01:11.924619 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:12.574753 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282357.sst): second pass time = 4.62's, 113.858'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3337.978'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.275'MB/sec write SST data time = 0.18's, 946.736'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.273'MB, throughput = 153531.186'MB/sec zip my value time = 4.62's, unzip length = 0.527'GB zip my value throughput = 113.858'MB/sec zip pipeline throughput = 113.858'MB/sec entries = 517713 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 usrkeys = 517713 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 17.272892 GB avg = 0.285 KB (by entry num) total key len = 1.606490 GB avg = 0.026 KB total ukey len = 1.097280 GB avg = 0.015 KB total ukey num = 0.073833896 Billion total entry num = 0.060641625 Billion write speed all = 0.400323118 MB/sec (with seq num) write speed all = 0.390036238 MB/sec (without seq num) 2024/11/23-22:01:12.789919 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282357: 517713 keys, 170303340 bytes, temperature: kWarm 2024/11/23-22:01:12.790009 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370472789956, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282357, "file_size": 170303340, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164156880, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354260, "raw_average_key_size": 20, "raw_value_size": 526517040, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517713, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370464, "oldest_key_time": 0, "file_creation_time": 1732370464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:12.887757 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst) /dev/shm/ToplingTemp/Topling-199889-U1lzUR 2024/11/23-22:01:15.533883 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst): first pass time = 2.65's, 202.893'MB/sec 2024/11/23-22:01:20.166440 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst): old prealloc_size = 590558003, real_size = 165162992 2024/11/23-22:01:20.166747 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517712 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.09 2024/11/23-22:01:20.166786 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:20.808375 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282358.sst): second pass time = 4.63's, 113.704'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 265.315'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.646'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.279'MB/sec write SST data time = 0.18's, 959.091'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 149427.033'MB/sec zip my value time = 4.63's, unzip length = 0.527'GB zip my value throughput = 113.704'MB/sec zip pipeline throughput = 113.704'MB/sec entries = 517712 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 usrkeys = 517712 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2073 dict = 1.00 all = 3.1523 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 17.799409 GB avg = 0.291 KB (by entry num) total key len = 1.616845 GB avg = 0.026 KB total ukey len = 1.103492 GB avg = 0.015 KB total ukey num = 0.074351608 Billion total entry num = 0.061159337 Billion write speed all = 0.411635197 MB/sec (with seq num) write speed all = 0.401262306 MB/sec (without seq num) 2024/11/23-22:01:21.005341 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282358: 517712 keys, 170309022 bytes, temperature: kWarm 2024/11/23-22:01:21.005405 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370481005372, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282358, "file_size": 170309022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164162848, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354240, "raw_average_key_size": 20, "raw_value_size": 526516707, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370472, "oldest_key_time": 0, "file_creation_time": 1732370472, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:21.099636 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst) /dev/shm/ToplingTemp/Topling-199889-up55rQ 2024/11/23-22:01:23.770176 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst): first pass time = 2.67's, 201.038'MB/sec 2024/11/23-22:01:28.462596 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst): old prealloc_size = 590558003, real_size = 165159600 2024/11/23-22:01:28.462916 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517717 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.08 2024/11/23-22:01:28.462952 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:29.105704 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282359.sst): second pass time = 4.69's, 112.258'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 264.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3102.126'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.279'MB/sec write SST data time = 0.18's, 952.762'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 153170.211'MB/sec zip my value time = 4.69's, unzip length = 0.527'GB zip my value throughput = 112.258'MB/sec zip pipeline throughput = 112.258'MB/sec entries = 517717 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 usrkeys = 517717 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2073 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 18.325926 GB avg = 0.297 KB (by entry num) total key len = 1.627199 GB avg = 0.026 KB total ukey len = 1.109705 GB avg = 0.015 KB total ukey num = 0.074869325 Billion total entry num = 0.061677054 Billion write speed all = 0.422942758 MB/sec (with seq num) write speed all = 0.412483900 MB/sec (without seq num) 2024/11/23-22:01:29.299297 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282359: 517717 keys, 170305821 bytes, temperature: kWarm 2024/11/23-22:01:29.299369 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370489299332, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282359, "file_size": 170305821, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164159456, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354340, "raw_average_key_size": 20, "raw_value_size": 526516721, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517717, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370481, "oldest_key_time": 0, "file_creation_time": 1732370481, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:29.408181 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst) /dev/shm/ToplingTemp/Topling-199889-TECLoS 2024/11/23-22:01:32.149722 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst): first pass time = 2.74's, 195.832'MB/sec 2024/11/23-22:01:36.660485 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst): old prealloc_size = 590558003, real_size = 165162368 2024/11/23-22:01:36.660794 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517709 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.09 2024/11/23-22:01:36.660838 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:37.307307 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282360.sst): second pass time = 4.51's, 116.768'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 263.308'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.103'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.277'MB/sec write SST data time = 0.18's, 948.025'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.271'MB, throughput = 151460.924'MB/sec zip my value time = 4.51's, unzip length = 0.527'GB zip my value throughput = 116.768'MB/sec zip pipeline throughput = 116.768'MB/sec entries = 517709 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 usrkeys = 517709 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2073 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 18.852443 GB avg = 0.303 KB (by entry num) total key len = 1.637553 GB avg = 0.026 KB total ukey len = 1.115917 GB avg = 0.015 KB total ukey num = 0.075387034 Billion total entry num = 0.062194763 Billion write speed all = 0.434247228 MB/sec (with seq num) write speed all = 0.423702412 MB/sec (without seq num) 2024/11/23-22:01:37.501076 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282360: 517709 keys, 170307594 bytes, temperature: kWarm 2024/11/23-22:01:37.501157 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370497501111, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282360, "file_size": 170307594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164162224, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354180, "raw_average_key_size": 20, "raw_value_size": 526517073, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370489, "oldest_key_time": 0, "file_creation_time": 1732370489, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:37.599926 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst) /dev/shm/ToplingTemp/Topling-199889-ccO8IP 2024/11/23-22:01:40.354558 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst): first pass time = 2.75's, 194.901'MB/sec 2024/11/23-22:01:44.955956 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst): old prealloc_size = 590558003, real_size = 165150432 2024/11/23-22:01:44.956306 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517716 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-22:01:44.956349 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:45.605198 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282361.sst): second pass time = 4.60's, 114.477'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 262.311'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3007.629'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 944.860'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 159238.695'MB/sec zip my value time = 4.60's, unzip length = 0.527'GB zip my value throughput = 114.477'MB/sec zip pipeline throughput = 114.477'MB/sec entries = 517716 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517716 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1526 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 19.378960 GB avg = 0.309 KB (by entry num) total key len = 1.647907 GB avg = 0.026 KB total ukey len = 1.122130 GB avg = 0.015 KB total ukey num = 0.075904750 Billion total entry num = 0.062712479 Billion write speed all = 0.445546873 MB/sec (with seq num) write speed all = 0.434916150 MB/sec (without seq num) 2024/11/23-22:01:45.798273 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282361: 517716 keys, 170295777 bytes, temperature: kWarm 2024/11/23-22:01:45.798346 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370505798309, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282361, "file_size": 170295777, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164150288, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354320, "raw_average_key_size": 20, "raw_value_size": 526517571, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370497, "oldest_key_time": 0, "file_creation_time": 1732370497, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:45.897205 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst) /dev/shm/ToplingTemp/Topling-199889-OcJafS 2024/11/23-22:01:48.672398 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst): first pass time = 2.78's, 193.457'MB/sec 2024/11/23-22:01:53.299715 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst): old prealloc_size = 590558003, real_size = 165157536 2024/11/23-22:01:53.300053 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517713 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.01 avg-zval = 317.08 2024/11/23-22:01:53.300095 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:01:53.943536 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282362.sst): second pass time = 4.63's, 113.829'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 264.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3053.049'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.278'MB/sec write SST data time = 0.18's, 955.309'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 153612.617'MB/sec zip my value time = 4.63's, unzip length = 0.527'GB zip my value throughput = 113.829'MB/sec zip pipeline throughput = 113.829'MB/sec entries = 517713 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 usrkeys = 517713 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.01 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2074 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 19.905477 GB avg = 0.315 KB (by entry num) total key len = 1.658262 GB avg = 0.026 KB total ukey len = 1.128342 GB avg = 0.015 KB total ukey num = 0.076422463 Billion total entry num = 0.063230192 Billion write speed all = 0.456842136 MB/sec (with seq num) write speed all = 0.446125547 MB/sec (without seq num) 2024/11/23-22:01:54.130193 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282362: 517713 keys, 170303269 bytes, temperature: kWarm 2024/11/23-22:01:54.130266 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370514130230, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282362, "file_size": 170303269, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164157392, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354260, "raw_average_key_size": 20, "raw_value_size": 526516811, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517713, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370505, "oldest_key_time": 0, "file_creation_time": 1732370505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:01:54.220847 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst) /dev/shm/ToplingTemp/Topling-199889-edmOoT 2024/11/23-22:01:56.861008 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst): first pass time = 2.64's, 203.351'MB/sec 2024/11/23-22:02:01.434530 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst): old prealloc_size = 590558003, real_size = 165154096 2024/11/23-22:02:01.434878 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517716 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-22:02:01.434918 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:02:02.096350 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282363.sst): second pass time = 4.57's, 115.166'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 257.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2883.183'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.271'MB/sec write SST data time = 0.18's, 928.519'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 136222.825'MB/sec zip my value time = 4.57's, unzip length = 0.527'GB zip my value throughput = 115.166'MB/sec zip pipeline throughput = 115.166'MB/sec entries = 517716 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517716 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 20.431994 GB avg = 0.321 KB (by entry num) total key len = 1.668616 GB avg = 0.026 KB total ukey len = 1.134555 GB avg = 0.015 KB total ukey num = 0.076940179 Billion total entry num = 0.063747908 Billion write speed all = 0.468135253 MB/sec (with seq num) write speed all = 0.457332784 MB/sec (without seq num) 2024/11/23-22:02:02.290703 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282363: 517716 keys, 170300124 bytes, temperature: kWarm 2024/11/23-22:02:02.290777 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370522290739, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282363, "file_size": 170300124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164153952, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354320, "raw_average_key_size": 20, "raw_value_size": 526516930, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 517716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370514, "oldest_key_time": 0, "file_creation_time": 1732370514, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:02.384226 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst) /dev/shm/ToplingTemp/Topling-199889-xejtQQ 2024/11/23-22:02:05.180249 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst): first pass time = 2.80's, 192.016'MB/sec 2024/11/23-22:02:09.652289 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst): old prealloc_size = 590558003, real_size = 165162128 2024/11/23-22:02:09.652586 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517708 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.02 avg-zval = 317.09 2024/11/23-22:02:09.652624 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:02:10.303547 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282364.sst): second pass time = 4.47's, 117.782'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 261.514'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3406.555'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.276'MB/sec write SST data time = 0.18's, 936.938'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 147290.584'MB/sec zip my value time = 4.47's, unzip length = 0.527'GB zip my value throughput = 117.782'MB/sec zip pipeline throughput = 117.782'MB/sec entries = 517708 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.09 usrkeys = 517708 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.02 avg-zval = 317.09 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1365 value = 3.2073 dict = 1.00 all = 3.1524 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 20.958511 GB avg = 0.326 KB (by entry num) total key len = 1.678970 GB avg = 0.026 KB total ukey len = 1.140767 GB avg = 0.015 KB total ukey num = 0.077457887 Billion total entry num = 0.064265616 Billion write speed all = 0.479423923 MB/sec (with seq num) write speed all = 0.468535619 MB/sec (without seq num) 2024/11/23-22:02:10.492194 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282364: 517708 keys, 170307422 bytes, temperature: kWarm 2024/11/23-22:02:10.492264 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370530492229, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282364, "file_size": 170307422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164161984, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354160, "raw_average_key_size": 20, "raw_value_size": 526517382, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517708, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370522, "oldest_key_time": 0, "file_creation_time": 1732370522, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:10.587340 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst) /dev/shm/ToplingTemp/Topling-199889-EKJzqT 2024/11/23-22:02:13.212772 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst): first pass time = 2.63's, 204.492'MB/sec 2024/11/23-22:02:17.832548 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst): old prealloc_size = 590558003, real_size = 165842048 2024/11/23-22:02:17.832855 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517714 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1648 GB avg-val =1017.00 avg-zval = 318.40 2024/11/23-22:02:17.832895 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:02:18.479261 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282365.sst): second pass time = 4.62's, 114.019'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 264.404'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3302.145'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.278'MB/sec write SST data time = 0.18's, 946.445'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 160562.161'MB/sec zip my value time = 4.62's, unzip length = 0.527'GB zip my value throughput = 114.019'MB/sec zip pipeline throughput = 114.019'MB/sec entries = 517714 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 318.40 usrkeys = 517714 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 318.40 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1648 GB dict = 5.27 MB all = 0.1710 GB } UnZip/Zip{ index = 7.1366 value = 3.1941 dict = 1.00 all = 3.1398 } Zip/UnZip{ index = 0.1401 value = 0.3131 dict = 1.00 all = 0.3185 } ---------------------------- total value len = 21.485028 GB avg = 0.332 KB (by entry num) total key len = 1.689324 GB avg = 0.026 KB total ukey len = 1.146980 GB avg = 0.015 KB total ukey num = 0.077975601 Billion total entry num = 0.064783330 Billion write speed all = 0.490708986 MB/sec (with seq num) write speed all = 0.479734867 MB/sec (without seq num) 2024/11/23-22:02:18.664757 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282365: 517714 keys, 170987726 bytes, temperature: kWarm 2024/11/23-22:02:18.664829 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370538664793, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282365, "file_size": 170987726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164841904, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354280, "raw_average_key_size": 20, "raw_value_size": 526516719, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370530, "oldest_key_time": 0, "file_creation_time": 1732370530, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:18.757475 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst) /dev/shm/ToplingTemp/Topling-199889-Z6ifEQ 2024/11/23-22:02:21.537733 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst): first pass time = 2.78's, 193.104'MB/sec 2024/11/23-22:02:26.244875 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst): old prealloc_size = 590558003, real_size = 165152656 2024/11/23-22:02:26.245194 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517715 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.07 2024/11/23-22:02:26.245225 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:02:26.913016 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282366.sst): second pass time = 4.70's, 111.923'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 254.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2998.367'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.266'MB/sec write SST data time = 0.18's, 936.544'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 160318.362'MB/sec zip my value time = 4.70's, unzip length = 0.527'GB zip my value throughput = 111.923'MB/sec zip pipeline throughput = 111.923'MB/sec entries = 517715 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 usrkeys = 517715 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2075 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 22.011545 GB avg = 0.337 KB (by entry num) total key len = 1.699679 GB avg = 0.026 KB total ukey len = 1.153193 GB avg = 0.015 KB total ukey num = 0.078493316 Billion total entry num = 0.065301045 Billion write speed all = 0.501987401 MB/sec (with seq num) write speed all = 0.490927558 MB/sec (without seq num) 2024/11/23-22:02:27.101207 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282366: 517715 keys, 170298329 bytes, temperature: kWarm 2024/11/23-22:02:27.101263 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370547101231, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282366, "file_size": 170298329, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164152512, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354300, "raw_average_key_size": 20, "raw_value_size": 526516745, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517715, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370538, "oldest_key_time": 0, "file_creation_time": 1732370538, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:27.198349 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst) /dev/shm/ToplingTemp/Topling-199889-m9lk4T 2024/11/23-22:02:29.606291 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst): first pass time = 2.41's, 222.961'MB/sec 2024/11/23-22:02:35.913405 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst): old prealloc_size = 590558003, real_size = 165155264 2024/11/23-22:02:35.913712 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 517714 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 1.68 raw-val = 0.5265 GB zip-val = 0.1642 GB avg-val =1017.00 avg-zval = 317.08 2024/11/23-22:02:35.913752 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/11/23-22:02:36.570999 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282367.sst): second pass time = 6.31's, 83.508'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 258.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.679'MB/sec (index lex order gen) rebuild zvType time = 0.48's, 0.269'MB/sec write SST data time = 0.18's, 967.565'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.272'MB, throughput = 84451.435'MB/sec zip my value time = 6.31's, unzip length = 0.527'GB zip my value throughput = 83.508'MB/sec zip pipeline throughput = 104.783'MB/sec entries = 517714 avg-key = 20.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 usrkeys = 517714 avg-key = 12.00 avg-zkey = 1.68 avg-val = 1017.00 avg-zval = 317.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0062 GB value = 0.5265 GB dict = 5.27 MB all = 0.5369 GB } __ZipSize{ index = 0.0009 GB value = 0.1642 GB dict = 5.27 MB all = 0.1703 GB } UnZip/Zip{ index = 7.1366 value = 3.2074 dict = 1.00 all = 3.1525 } Zip/UnZip{ index = 0.1401 value = 0.3118 dict = 1.00 all = 0.3172 } ---------------------------- total value len = 22.538062 GB avg = 0.342 KB (by entry num) total key len = 1.710033 GB avg = 0.026 KB total ukey len = 1.159405 GB avg = 0.015 KB total ukey num = 0.079011030 Billion total entry num = 0.065818759 Billion write speed all = 0.513248489 MB/sec (with seq num) write speed all = 0.502103240 MB/sec (without seq num) 2024/11/23-22:02:36.776444 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282367: 517714 keys, 170301516 bytes, temperature: kWarm 2024/11/23-22:02:36.776514 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370556776478, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282367, "file_size": 170301516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164155120, "index_size": 870528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10354280, "raw_average_key_size": 20, "raw_value_size": 526516724, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 517714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370547, "oldest_key_time": 0, "file_creation_time": 1732370547, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:36.872098 139957601228352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst) /dev/shm/ToplingTemp/Topling-199889-ACVwNR 2024/11/23-22:02:37.004681 139957601228352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst): first pass time = 0.13's, 737.036'MB/sec 2024/11/23-22:02:37.639686 139957601228352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst): old prealloc_size = 590558003, real_size = 29445392 2024/11/23-22:02:37.639787 139957601228352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 94200 prefix = 4 raw-key = 0.0008 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.0958 GB zip-val = 0.0293 GB avg-val =1017.02 avg-zval = 310.64 2024/11/23-22:02:37.639825 139957601228352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.098 GB 2024/11/23-22:02:37.757655 139957601228352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00/282368.sst): second pass time = 0.63's, 151.035'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 257.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2026.062'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.274'MB/sec write SST data time = 0.03's, 953.111'MB/sec dict compress time = 3922768807.74's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.994'MB, throughput = 31482.445'MB/sec zip my value time = 0.63's, unzip length = 0.096'GB zip my value throughput = 151.035'MB/sec zip pipeline throughput = 151.035'MB/sec entries = 94200 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.02 avg-zval = 310.64 usrkeys = 94200 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1017.02 avg-zval = 310.64 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 7 UnZipSize{ index = 0.0011 GB value = 0.0958 GB dict = 0.99 MB all = 0.0977 GB } __ZipSize{ index = 0.0002 GB value = 0.0293 GB dict = 0.99 MB all = 0.0304 GB } UnZip/Zip{ index = 7.0792 value = 3.2740 dict = 1.00 all = 3.2115 } Zip/UnZip{ index = 0.1413 value = 0.3054 dict = 1.00 all = 0.3114 } ---------------------------- total value len = 22.633865 GB avg = 0.343 KB (by entry num) total key len = 1.711917 GB avg = 0.026 KB total ukey len = 1.160536 GB avg = 0.015 KB total ukey num = 0.079105230 Billion total entry num = 0.065912959 Billion write speed all = 0.515303245 MB/sec (with seq num) write speed all = 0.504142327 MB/sec (without seq num) 2024/11/23-22:02:37.800721 139957601228352 [compaction_job.cc:1948] [default] [JOB 36608] Generated table #282368: 94200 keys, 30417717 bytes, temperature: kWarm 2024/11/23-22:02:37.800790 139957601228352 EVENT_LOG_v1 {"time_micros": 1732370557800755, "cf_name": "default", "job": 36608, "event": "table_file_creation", "file_number": 282368, "file_size": 30417717, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 29261952, "index_size": 159680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1884000, "raw_average_key_size": 20, "raw_value_size": 95803165, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 94200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732370556, "oldest_key_time": 0, "file_creation_time": 1732370556, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 282368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-22:02:37.823669 139957601228352 [compaction_job.cc:703] job-36608: subcompact[0], size: 4.460669 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-36608/att-00 [282342,282343,282344,282345,282346,282347,282348,282349,282350,282351,282352,282353,282354,282355,282356,282357,282358,282359,282360,282361,282362,282363,282364,282365,282366,282367,282368] 2024/11/23-22:02:38.327245 139957601228352 (Original Log Time 2024/11/23-21:58:55.498305) EVENT_LOG_v1 {"time_micros": 1732370335498256, "job": 36608, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [281393, 281394, 281395, 281396, 281397, 281398, 281399, 281400, 281401, 281402, 281403, 281404, 281405, 281406, 281407, 281408, 281409, 281410, 281411, 281412, 281606, 281607, 281608, 281609, 281610, 281611, 281612, 281613, 281614, 281615, 281616, 281617, 281618, 281619, 281620, 281621, 281622, 281623, 281624, 281625, 281626, 281627, 281628, 281629, 281630, 281631, 281632, 281633, 281634, 281635, 281636, 281637, 281638], "score": -1, "input_data_size": 4354052931, "oldest_snapshot_seqno": 4286761365} 2024/11/23-22:02:38.327251 139957601228352 (Original Log Time 2024/11/23-22:02:38.322076) [compaction_job.cc:2022] [default] [JOB 36608] Compacted 53@4 files to L5 => 4460668930 bytes 2024/11/23-22:02:38.327254 139957601228352 (Original Log Time 2024/11/23-22:02:38.327159) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 0 0 27 0] max score 0.00, MB/sec: 19.6 rd, 20.1 wr, level 5, files in(53, 0) out(27 +0 blob) MB in(4152.3, 0.0 +0.0 blob) out(4254.0 +0.0 blob), read-write-amplify(2.0) write-amplify(1.0) OK, records in: 13554795, records dropped: 0 output_compression: Snappy 2024/11/23-22:02:38.327260 139957601228352 (Original Log Time 2024/11/23-22:02:38.327214) EVENT_LOG_v1 {"time_micros": 1732370558327186, "cf": "default", "job": 36608, "event": "compaction_finished", "compaction_time_micros": 222325375, "compaction_time_cpu_micros": 37244673, "output_level": 5, "num_output_files": 27, "total_output_size": 4460668930, "num_input_records": 13554795, "num_output_records": 13554795, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2867487277, "file_range_sync_nanos": 0, "file_fsync_nanos": 18846, "file_prepare_write_nanos": 73139, "lsm_state": [0, 0, 0, 0, 0, 27, 0]} 2024/11/23-22:02:38.457909 139957601228352 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732370335,"executesMs":222976,"compactionJobId":36608,"attempt":0,"compactionInputRawBytes":14056340774,"compactionInputZipBytes":4354052931,"compactionOutputRawBytes":0,"compactionOutputZipBytes":4460668930,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8a04b588e85669e4ead54857d8e2dab8-549672b8c39aa144-00"}}, response = 2024/11/23-22:02:38.458015 139957601228352 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-36608/att-00: olev 5, work 222.976 s, result 7.845 ms, install 5.204 ms, input{raw 13.091 GiB zip 4.055 GiB}