2024/11/24-02:10:52.872728 139957265520192 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00: fork to child time = 0.007856 sec 2024/11/24-02:10:52.873554 139957265520192 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00 2024/11/24-02:10:52.879201 139957265520192 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00 2024/11/24-02:10:52.880097 139957265520192 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-02:10:52.880451 139957265520192 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-02:10:52.935172 139957265520192 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39887/att-00/MANIFEST-206347 2024/11/24-02:10:52.963013 139957265520192 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39887/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 288352, last_sequence is 4388893151, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 288191 2024/11/24-02:10:52.963035 139957265520192 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-02:10:52.963182 139957265520192 [version_set.cc:5721] Creating manifest 288352 2024/11/24-02:10:52.997132 139957265520192 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-02:10:53.015953 139957265520192 [compaction_job.cc:2437] [default] [JOB 39887] Compacting 1@2 + 16@3 files to L3, score 0.67, subcompactions 1 : 1 2024/11/24-02:10:53.015988 139957265520192 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29883 Base level 2, inputs: [288316(27MB)], [288280(35MB) 288281(35MB) 288282(31MB) 288283(35MB) 288284(31MB) 288285(35MB) 288286(33MB) 288287(33MB) 288288(35MB) 288289(12MB) 288250(35MB) 288251(33MB) 288252(34MB) 288253(34MB) 288254(33MB) 288347(35MB)] 2024/11/24-02:10:53.022719 139957265520192 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:10:53.022752 139957265520192 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:10:53.022772 139957265520192 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:10:53.022778 139957265520192 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-02:10:53.022790 139957265520192 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-02:10:53.830066 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst) /dev/shm/ToplingTemp/Topling-245268-N3Hstb 2024/11/24-02:10:54.377244 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst): first pass time = 0.55's, 245.309'MB/sec 2024/11/24-02:10:55.842089 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst): old prealloc_size = 147639500, real_size = 33248672 2024/11/24-02:10:55.842631 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926146 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.92 avg-zval = 33.75 2024/11/24-02:10:55.843419 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:10:56.692073 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288353.sst): second pass time = 1.46's, 73.968'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3393.926'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 292.969'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27683.995'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.968'MB/sec zip pipeline throughput = 73.968'MB/sec entries = 926146 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 33.75 usrkeys = 926146 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 33.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926146 bytes = 3125760 } seq expand size = 6483022 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5152 value = 3.4645 dict = 1.00 all = 3.6052 } Zip/UnZip{ index = 0.0951 value = 0.2886 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 2251.440342 GB avg = 0.200 KB (by entry num) total key len = 306.908895 GB avg = 0.027 KB total ukey len = 237.373250 GB avg = 0.015 KB total ukey num = 15.739587309 Billion total entry num = 11.280901257 Billion write speed all = 41.167764909 MB/sec (with seq num) write speed all = 39.715548818 MB/sec (without seq num) 2024/11/24-02:10:56.738614 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288353: 926146 keys, 37229286 bytes, temperature: kWarm 2024/11/24-02:10:56.738708 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385456738649, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288353, "file_size": 37229286, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4388771325, "table_properties": {"data_size": 31255440, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932088, "raw_average_key_size": 28, "raw_value_size": 108285644, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926146, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385453, "oldest_key_time": 0, "file_creation_time": 1732385453, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:10:56.783820 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst) /dev/shm/ToplingTemp/Topling-245268-cYeUSc 2024/11/24-02:10:57.730731 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst): first pass time = 0.95's, 141.747'MB/sec 2024/11/24-02:10:59.211193 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst): old prealloc_size = 147639500, real_size = 33167680 2024/11/24-02:10:59.211724 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926222 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.91 avg-zval = 33.64 2024/11/24-02:10:59.212537 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:00.051601 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288354.sst): second pass time = 1.48's, 73.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.209'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3491.037'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 290.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34414.665'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.203'MB/sec zip pipeline throughput = 73.203'MB/sec entries = 926222 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 33.64 usrkeys = 926222 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.91 avg-zval = 33.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926222 bytes = 3126016 } seq expand size = 6483554 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4444 value = 3.4748 dict = 1.00 all = 3.6126 } Zip/UnZip{ index = 0.0957 value = 0.2878 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 2251.555109 GB avg = 0.200 KB (by entry num) total key len = 306.934829 GB avg = 0.027 KB total ukey len = 237.391775 GB avg = 0.015 KB total ukey num = 15.740513531 Billion total entry num = 11.281827479 Billion write speed all = 41.167803476 MB/sec (with seq num) write speed all = 39.715546659 MB/sec (without seq num) 2024/11/24-02:11:00.096276 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288354: 926222 keys, 37152366 bytes, temperature: kWarm 2024/11/24-02:11:00.096347 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385460096312, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288354, "file_size": 37152366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4388771393, "table_properties": {"data_size": 31162288, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934216, "raw_average_key_size": 28, "raw_value_size": 108283512, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926222, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385456, "oldest_key_time": 0, "file_creation_time": 1732385456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:00.133113 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst) /dev/shm/ToplingTemp/Topling-245268-FA05ie 2024/11/24-02:11:01.081911 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst): first pass time = 0.95's, 141.467'MB/sec 2024/11/24-02:11:02.483287 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst): old prealloc_size = 147639500, real_size = 29077472 2024/11/24-02:11:02.483855 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938597 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1079 GB zip-val = 0.0270 GB avg-val = 115.00 avg-zval = 28.80 2024/11/24-02:11:02.484642 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:03.321225 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288355.sst): second pass time = 1.40's, 77.074'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.493'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.849'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.331'MB/sec write SST data time = 0.13's, 257.781'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34391.940'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.074'MB/sec zip pipeline throughput = 77.074'MB/sec entries = 938597 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.00 avg-zval = 28.80 usrkeys = 938597 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.00 avg-zval = 28.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938597 bytes = 3167776 } seq expand size = 6570179 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0270 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.3765 value = 3.9927 dict = 1.00 all = 4.0559 } Zip/UnZip{ index = 0.0964 value = 0.2505 dict = 1.00 all = 0.2466 } ---------------------------- total value len = 2251.669616 GB avg = 0.200 KB (by entry num) total key len = 306.961110 GB avg = 0.027 KB total ukey len = 237.410547 GB avg = 0.015 KB total ukey num = 15.741452128 Billion total entry num = 11.282766076 Billion write speed all = 41.167902984 MB/sec (with seq num) write speed all = 39.715601753 MB/sec (without seq num) 2024/11/24-02:11:03.362650 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288355: 938597 keys, 33091626 bytes, temperature: kWarm 2024/11/24-02:11:03.362723 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385463362686, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288355, "file_size": 33091626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4388770693, "table_properties": {"data_size": 27033552, "index_size": 1809088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26280716, "raw_average_key_size": 28, "raw_value_size": 107937046, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938597, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385460, "oldest_key_time": 0, "file_creation_time": 1732385460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:03.397796 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst) /dev/shm/ToplingTemp/Topling-245268-dHB9fd 2024/11/24-02:11:04.416805 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst): first pass time = 1.02's, 131.720'MB/sec 2024/11/24-02:11:05.889321 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst): old prealloc_size = 147639500, real_size = 32762320 2024/11/24-02:11:05.889832 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931699 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.06 avg-zval = 33.01 2024/11/24-02:11:05.890605 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:06.750347 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288356.sst): second pass time = 1.47's, 73.498'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.692'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3622.622'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.12's, 298.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29105.771'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.498'MB/sec zip pipeline throughput = 73.498'MB/sec entries = 931699 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 33.01 usrkeys = 931699 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 33.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931699 bytes = 3144496 } seq expand size = 6521893 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5118 value = 3.5157 dict = 1.00 all = 3.6514 } Zip/UnZip{ index = 0.0951 value = 0.2844 dict = 1.00 all = 0.2739 } ---------------------------- total value len = 2251.784268 GB avg = 0.200 KB (by entry num) total key len = 306.987197 GB avg = 0.027 KB total ukey len = 237.429181 GB avg = 0.015 KB total ukey num = 15.742383827 Billion total entry num = 11.283697775 Billion write speed all = 41.167896065 MB/sec (with seq num) write speed all = 39.715555038 MB/sec (without seq num) 2024/11/24-02:11:06.796843 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288356: 931699 keys, 36757570 bytes, temperature: kWarm 2024/11/24-02:11:06.796909 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385466796874, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288356, "file_size": 36757570, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4388769819, "table_properties": {"data_size": 30756544, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087572, "raw_average_key_size": 28, "raw_value_size": 108130214, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931699, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385463, "oldest_key_time": 0, "file_creation_time": 1732385463, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:06.833232 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst) /dev/shm/ToplingTemp/Topling-245268-dcafId 2024/11/24-02:11:07.809634 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst): first pass time = 0.98's, 137.466'MB/sec 2024/11/24-02:11:09.090941 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst): old prealloc_size = 147639500, real_size = 29359696 2024/11/24-02:11:09.091497 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936246 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0273 GB avg-val = 115.36 avg-zval = 29.19 2024/11/24-02:11:09.092266 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:09.923462 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288357.sst): second pass time = 1.28's, 84.342'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3524.382'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 258.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32557.510'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.342'MB/sec zip pipeline throughput = 84.342'MB/sec entries = 936246 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.36 avg-zval = 29.19 usrkeys = 936246 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.36 avg-zval = 29.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936246 bytes = 3159840 } seq expand size = 6553722 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0273 GB dict = 1.08 MB all = 0.0334 GB } UnZip/Zip{ index = 10.4443 value = 3.9514 dict = 1.00 all = 4.0225 } Zip/UnZip{ index = 0.0957 value = 0.2531 dict = 1.00 all = 0.2486 } ---------------------------- total value len = 2251.898825 GB avg = 0.200 KB (by entry num) total key len = 307.013412 GB avg = 0.027 KB total ukey len = 237.447906 GB avg = 0.015 KB total ukey num = 15.743320073 Billion total entry num = 11.284634021 Billion write speed all = 41.168059215 MB/sec (with seq num) write speed all = 39.715671830 MB/sec (without seq num) 2024/11/24-02:11:09.964124 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288357: 936246 keys, 33366752 bytes, temperature: kWarm 2024/11/24-02:11:09.964190 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385469964152, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288357, "file_size": 33366752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4291161237, "largest_seqno": 4388770703, "table_properties": {"data_size": 27332608, "index_size": 1792832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26214888, "raw_average_key_size": 28, "raw_value_size": 108002900, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936246, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385466, "oldest_key_time": 0, "file_creation_time": 1732385466, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:10.007401 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst) /dev/shm/ToplingTemp/Topling-245268-LJXAAc 2024/11/24-02:11:10.870367 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst): first pass time = 0.86's, 155.537'MB/sec 2024/11/24-02:11:12.206145 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst): old prealloc_size = 147639500, real_size = 33014592 2024/11/24-02:11:12.206665 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927260 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.75 avg-zval = 33.45 2024/11/24-02:11:12.207429 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:13.060393 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288358.sst): second pass time = 1.33's, 81.093'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.312'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3545.834'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 294.710'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32704.109'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.093'MB/sec zip pipeline throughput = 81.093'MB/sec entries = 927260 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.45 usrkeys = 927260 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927260 bytes = 3129520 } seq expand size = 6490820 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5145 value = 3.4900 dict = 1.00 all = 3.6277 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 2252.013570 GB avg = 0.200 KB (by entry num) total key len = 307.039375 GB avg = 0.027 KB total ukey len = 237.466451 GB avg = 0.015 KB total ukey num = 15.744247333 Billion total entry num = 11.285561281 Billion write speed all = 41.168245301 MB/sec (with seq num) write speed all = 39.715811873 MB/sec (without seq num) 2024/11/24-02:11:13.117190 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288358: 927260 keys, 36998280 bytes, temperature: kWarm 2024/11/24-02:11:13.117261 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385473117225, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288358, "file_size": 36998280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4388768290, "table_properties": {"data_size": 31018800, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963280, "raw_average_key_size": 28, "raw_value_size": 108254520, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927260, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385469, "oldest_key_time": 0, "file_creation_time": 1732385469, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:13.157780 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst) /dev/shm/ToplingTemp/Topling-245268-3JJaZd 2024/11/24-02:11:14.159387 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst): first pass time = 1.00's, 134.008'MB/sec 2024/11/24-02:11:15.463155 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst): old prealloc_size = 147639500, real_size = 30669072 2024/11/24-02:11:15.463715 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933500 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 115.78 avg-zval = 30.70 2024/11/24-02:11:15.464484 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:16.290990 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288359.sst): second pass time = 1.30's, 82.947'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.883'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3316.203'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.12's, 281.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32662.123'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.947'MB/sec zip pipeline throughput = 82.947'MB/sec entries = 933500 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.78 avg-zval = 30.70 usrkeys = 933500 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.78 avg-zval = 30.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933500 bytes = 3150576 } seq expand size = 6534500 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.5128 value = 3.7712 dict = 1.00 all = 3.8712 } Zip/UnZip{ index = 0.0951 value = 0.2652 dict = 1.00 all = 0.2583 } ---------------------------- total value len = 2252.128184 GB avg = 0.200 KB (by entry num) total key len = 307.065513 GB avg = 0.027 KB total ukey len = 237.485121 GB avg = 0.015 KB total ukey num = 15.745180833 Billion total entry num = 11.286494781 Billion write speed all = 41.168370048 MB/sec (with seq num) write speed all = 39.715891967 MB/sec (without seq num) 2024/11/24-02:11:16.338021 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288359: 933500 keys, 34670484 bytes, temperature: kWarm 2024/11/24-02:11:16.338092 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385476338057, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288359, "file_size": 34670484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4388771504, "table_properties": {"data_size": 28659584, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26138000, "raw_average_key_size": 28, "raw_value_size": 108079760, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933500, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385473, "oldest_key_time": 0, "file_creation_time": 1732385473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:16.379145 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst) /dev/shm/ToplingTemp/Topling-245268-PJlHrc 2024/11/24-02:11:17.330926 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst): first pass time = 0.95's, 141.022'MB/sec 2024/11/24-02:11:18.611191 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst): old prealloc_size = 147639500, real_size = 31359984 2024/11/24-02:11:18.611704 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-02:11:18.613439 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:19.429480 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288360.sst): second pass time = 1.28's, 84.864'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.167'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3567.343'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 281.248'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 28619.171'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.864'MB/sec zip pipeline throughput = 84.864'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.6935 dict = 1.00 all = 3.7999 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2252.243097 GB avg = 0.200 KB (by entry num) total key len = 307.091254 GB avg = 0.027 KB total ukey len = 237.503507 GB avg = 0.015 KB total ukey num = 15.746100133 Billion total entry num = 11.287414081 Billion write speed all = 41.168554162 MB/sec (with seq num) write speed all = 39.716031109 MB/sec (without seq num) 2024/11/24-02:11:19.475536 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288360: 919300 keys, 35321842 bytes, temperature: kWarm 2024/11/24-02:11:19.475609 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385479475571, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288360, "file_size": 35321842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4274028011, "largest_seqno": 4383729209, "table_properties": {"data_size": 29369568, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385476, "oldest_key_time": 0, "file_creation_time": 1732385476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:19.513013 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst) /dev/shm/ToplingTemp/Topling-245268-PCzYte 2024/11/24-02:11:20.425757 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst): first pass time = 0.91's, 147.055'MB/sec 2024/11/24-02:11:21.765646 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst): old prealloc_size = 147639500, real_size = 32903984 2024/11/24-02:11:21.766213 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930581 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.23 avg-zval = 33.21 2024/11/24-02:11:21.767826 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:22.625642 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288361.sst): second pass time = 1.34's, 80.804'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.904'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.239'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 292.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31768.211'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.804'MB/sec zip pipeline throughput = 80.804'MB/sec entries = 930581 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.23 avg-zval = 33.21 usrkeys = 930581 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.23 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930581 bytes = 3140720 } seq expand size = 6514067 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5140 value = 3.5003 dict = 1.00 all = 3.6377 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2252.357773 GB avg = 0.200 KB (by entry num) total key len = 307.117310 GB avg = 0.027 KB total ukey len = 237.522118 GB avg = 0.015 KB total ukey num = 15.747030714 Billion total entry num = 11.288344662 Billion write speed all = 41.168701352 MB/sec (with seq num) write speed all = 39.716133227 MB/sec (without seq num) 2024/11/24-02:11:22.673311 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288361: 930581 keys, 36895958 bytes, temperature: kWarm 2024/11/24-02:11:22.673382 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385482673346, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288361, "file_size": 36895958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4388771723, "table_properties": {"data_size": 30900960, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26056268, "raw_average_key_size": 28, "raw_value_size": 108161542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930581, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385479, "oldest_key_time": 0, "file_creation_time": 1732385479, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:22.709348 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst) /dev/shm/ToplingTemp/Topling-245268-isa1Gd 2024/11/24-02:11:23.794136 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst): first pass time = 1.08's, 123.731'MB/sec 2024/11/24-02:11:25.084029 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst): old prealloc_size = 147639500, real_size = 30401008 2024/11/24-02:11:25.084566 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929174 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.45 avg-zval = 30.55 2024/11/24-02:11:25.085345 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:25.897337 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288362.sst): second pass time = 1.29's, 83.965'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.713'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.12's, 277.485'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31899.879'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.965'MB/sec zip pipeline throughput = 83.965'MB/sec entries = 929174 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.45 avg-zval = 30.55 usrkeys = 929174 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.45 avg-zval = 30.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929174 bytes = 3135984 } seq expand size = 6504218 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4464 value = 3.8113 dict = 1.00 all = 3.9030 } Zip/UnZip{ index = 0.0957 value = 0.2624 dict = 1.00 all = 0.2562 } ---------------------------- total value len = 2252.472478 GB avg = 0.200 KB (by entry num) total key len = 307.143327 GB avg = 0.027 KB total ukey len = 237.540702 GB avg = 0.015 KB total ukey num = 15.747959888 Billion total entry num = 11.289273836 Billion write speed all = 41.168798340 MB/sec (with seq num) write speed all = 39.716187094 MB/sec (without seq num) 2024/11/24-02:11:25.941200 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288362: 929174 keys, 34388148 bytes, temperature: kWarm 2024/11/24-02:11:25.941281 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385485941244, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288362, "file_size": 34388148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4320749715, "largest_seqno": 4388770749, "table_properties": {"data_size": 28389600, "index_size": 1778944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016872, "raw_average_key_size": 28, "raw_value_size": 108200856, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929174, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385482, "oldest_key_time": 0, "file_creation_time": 1732385482, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:25.980748 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst) /dev/shm/ToplingTemp/Topling-245268-DPOV4d 2024/11/24-02:11:26.892185 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst): first pass time = 0.91's, 147.266'MB/sec 2024/11/24-02:11:28.188841 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst): old prealloc_size = 147639500, real_size = 31650288 2024/11/24-02:11:28.189401 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919952 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.90 avg-zval = 32.25 2024/11/24-02:11:28.190172 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:29.001852 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288363.sst): second pass time = 1.30's, 83.694'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.808'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.683'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 285.814'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35048.669'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.694'MB/sec zip pipeline throughput = 83.694'MB/sec entries = 919952 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.90 avg-zval = 32.25 usrkeys = 919952 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.90 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919952 bytes = 3104848 } seq expand size = 6439664 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5133 value = 3.6555 dict = 1.00 all = 3.7685 } Zip/UnZip{ index = 0.0951 value = 0.2736 dict = 1.00 all = 0.2654 } ---------------------------- total value len = 2252.587376 GB avg = 0.200 KB (by entry num) total key len = 307.169085 GB avg = 0.027 KB total ukey len = 237.559101 GB avg = 0.015 KB total ukey num = 15.748879840 Billion total entry num = 11.290193788 Billion write speed all = 41.169004983 MB/sec (with seq num) write speed all = 39.716347900 MB/sec (without seq num) 2024/11/24-02:11:29.046091 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288363: 919952 keys, 35615506 bytes, temperature: kWarm 2024/11/24-02:11:29.046201 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385489046165, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288363, "file_size": 35615506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4388771177, "table_properties": {"data_size": 29670048, "index_size": 1750080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25758656, "raw_average_key_size": 28, "raw_value_size": 108459072, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919952, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385485, "oldest_key_time": 0, "file_creation_time": 1732385485, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:29.086065 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst) /dev/shm/ToplingTemp/Topling-245268-vAcJWc 2024/11/24-02:11:30.101419 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst): first pass time = 1.02's, 132.192'MB/sec 2024/11/24-02:11:31.424611 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst): old prealloc_size = 147639500, real_size = 32907632 2024/11/24-02:11:31.425171 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929853 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.34 avg-zval = 33.23 2024/11/24-02:11:31.425920 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:32.278321 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288364.sst): second pass time = 1.32's, 81.808'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.274'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 299.892'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33500.807'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.808'MB/sec zip pipeline throughput = 81.808'MB/sec entries = 929853 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.23 usrkeys = 929853 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.34 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929853 bytes = 3138272 } seq expand size = 6508971 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4446 value = 3.5017 dict = 1.00 all = 3.6375 } Zip/UnZip{ index = 0.0957 value = 0.2856 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2252.702067 GB avg = 0.200 KB (by entry num) total key len = 307.195121 GB avg = 0.027 KB total ukey len = 237.577698 GB avg = 0.015 KB total ukey num = 15.749809693 Billion total entry num = 11.291123641 Billion write speed all = 41.169098870 MB/sec (with seq num) write speed all = 39.716398698 MB/sec (without seq num) 2024/11/24-02:11:32.325060 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288364: 929853 keys, 36898478 bytes, temperature: kWarm 2024/11/24-02:11:32.325131 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385492325095, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288364, "file_size": 36898478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4388771828, "table_properties": {"data_size": 30894432, "index_size": 1780544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035884, "raw_average_key_size": 28, "raw_value_size": 108181954, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385489, "oldest_key_time": 0, "file_creation_time": 1732385489, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:32.361996 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst) /dev/shm/ToplingTemp/Topling-245268-2nxHnf 2024/11/24-02:11:33.188553 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst): first pass time = 0.83's, 162.390'MB/sec 2024/11/24-02:11:34.482941 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst): old prealloc_size = 147639500, real_size = 30734416 2024/11/24-02:11:34.483500 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926856 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.81 avg-zval = 30.99 2024/11/24-02:11:34.484248 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:35.295963 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288365.sst): second pass time = 1.29's, 83.739'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.701'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3425.766'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.12's, 278.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33940.129'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.739'MB/sec zip pipeline throughput = 83.739'MB/sec entries = 926856 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.81 avg-zval = 30.99 usrkeys = 926856 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.81 avg-zval = 30.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926856 bytes = 3128160 } seq expand size = 6487992 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4443 value = 3.7687 dict = 1.00 all = 3.8662 } Zip/UnZip{ index = 0.0957 value = 0.2653 dict = 1.00 all = 0.2586 } ---------------------------- total value len = 2252.816821 GB avg = 0.200 KB (by entry num) total key len = 307.221073 GB avg = 0.027 KB total ukey len = 237.596235 GB avg = 0.015 KB total ukey num = 15.750736549 Billion total entry num = 11.292050497 Billion write speed all = 41.169363769 MB/sec (with seq num) write speed all = 39.716614853 MB/sec (without seq num) 2024/11/24-02:11:35.340575 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288365: 926856 keys, 34715316 bytes, temperature: kWarm 2024/11/24-02:11:35.340646 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385495340611, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288365, "file_size": 34715316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4311042110, "largest_seqno": 4388770773, "table_properties": {"data_size": 28727680, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951968, "raw_average_key_size": 28, "raw_value_size": 108265848, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926856, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385492, "oldest_key_time": 0, "file_creation_time": 1732385492, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:35.394743 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst) /dev/shm/ToplingTemp/Topling-245268-HbIf1b 2024/11/24-02:11:36.314552 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst): first pass time = 0.92's, 145.926'MB/sec 2024/11/24-02:11:37.624474 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst): old prealloc_size = 147639500, real_size = 32413024 2024/11/24-02:11:37.625025 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922458 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.50 avg-zval = 32.98 2024/11/24-02:11:37.625763 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:38.447304 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288366.sst): second pass time = 1.31's, 82.798'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.220'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.500'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.12's, 293.059'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28979.263'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.798'MB/sec zip pipeline throughput = 82.798'MB/sec entries = 922458 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 32.98 usrkeys = 922458 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.50 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922458 bytes = 3113312 } seq expand size = 6457206 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5123 value = 3.5622 dict = 1.00 all = 3.6889 } Zip/UnZip{ index = 0.0951 value = 0.2807 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 2252.931667 GB avg = 0.199 KB (by entry num) total key len = 307.246902 GB avg = 0.027 KB total ukey len = 237.614684 GB avg = 0.015 KB total ukey num = 15.751659007 Billion total entry num = 11.292972955 Billion write speed all = 41.169539628 MB/sec (with seq num) write speed all = 39.716745661 MB/sec (without seq num) 2024/11/24-02:11:38.492444 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288366: 922458 keys, 36384562 bytes, temperature: kWarm 2024/11/24-02:11:38.492518 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385498492482, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288366, "file_size": 36384562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4388765156, "table_properties": {"data_size": 30427216, "index_size": 1755008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25828824, "raw_average_key_size": 28, "raw_value_size": 108388960, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922458, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385495, "oldest_key_time": 0, "file_creation_time": 1732385495, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:38.528218 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst) /dev/shm/ToplingTemp/Topling-245268-cR4p9a 2024/11/24-02:11:39.461254 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst): first pass time = 0.93's, 143.856'MB/sec 2024/11/24-02:11:40.788638 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst): old prealloc_size = 147639500, real_size = 32725472 2024/11/24-02:11:40.789242 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929984 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.32 avg-zval = 33.04 2024/11/24-02:11:40.790024 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:11:41.635823 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288367.sst): second pass time = 1.33's, 81.543'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2994.767'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32380.232'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.543'MB/sec zip pipeline throughput = 81.543'MB/sec entries = 929984 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.32 avg-zval = 33.04 usrkeys = 929984 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.32 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929984 bytes = 3138704 } seq expand size = 6509888 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5141 value = 3.5210 dict = 1.00 all = 3.6555 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 2253.046356 GB avg = 0.199 KB (by entry num) total key len = 307.272942 GB avg = 0.027 KB total ukey len = 237.633284 GB avg = 0.015 KB total ukey num = 15.752588991 Billion total entry num = 11.293902939 Billion write speed all = 41.169691711 MB/sec (with seq num) write speed all = 39.716852597 MB/sec (without seq num) 2024/11/24-02:11:41.680618 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288367: 929984 keys, 36716656 bytes, temperature: kWarm 2024/11/24-02:11:41.680688 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385501680654, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288367, "file_size": 36716656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4388770787, "table_properties": {"data_size": 30723792, "index_size": 1769024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26039552, "raw_average_key_size": 28, "raw_value_size": 108178308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929984, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385498, "oldest_key_time": 0, "file_creation_time": 1732385498, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:41.718517 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst) /dev/shm/ToplingTemp/Topling-245268-rWtlte 2024/11/24-02:11:41.964757 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst): first pass time = 0.25's, 249.757'MB/sec 2024/11/24-02:11:42.592203 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst): old prealloc_size = 147639500, real_size = 16125968 2024/11/24-02:11:42.592487 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 421173 prefix = 4 raw-key = 0.0067 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0497 GB zip-val = 0.0152 GB avg-val = 118.00 avg-zval = 36.11 2024/11/24-02:11:42.592838 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0010 GB, waited 0.000 sec, Key+Value = 0.061 GB 2024/11/24-02:11:42.981006 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00/288368.sst): second pass time = 0.63's, 79.293'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.39's, 46.033'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.110'MB/sec (index lex order gen) rebuild zvType time = 0.33's, 0.318'MB/sec write SST data time = 0.06's, 313.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.504'MB, throughput = 16451.775'MB/sec zip my value time = 0.63's, unzip length = 0.050'GB zip my value throughput = 79.293'MB/sec zip pipeline throughput = 79.293'MB/sec entries = 421173 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 36.11 usrkeys = 421173 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 36.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 421173 bytes = 1368832 } seq expand size = 2948211 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0084 GB value = 0.0497 GB dict = 0.50 MB all = 0.0615 GB } __ZipSize{ index = 0.0008 GB value = 0.0152 GB dict = 0.50 MB all = 0.0179 GB } UnZip/Zip{ index = 10.3643 value = 3.2680 dict = 1.00 all = 3.4361 } Zip/UnZip{ index = 0.0965 value = 0.3060 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 2253.099002 GB avg = 0.199 KB (by entry num) total key len = 307.284735 GB avg = 0.027 KB total ukey len = 237.641707 GB avg = 0.015 KB total ukey num = 15.753010164 Billion total entry num = 11.294324112 Billion write speed all = 41.169837382 MB/sec (with seq num) write speed all = 39.716975515 MB/sec (without seq num) 2024/11/24-02:11:43.008252 139957265520192 [compaction_job.cc:1948] [default] [JOB 39887] Generated table #288368: 421173 keys, 17895797 bytes, temperature: kWarm 2024/11/24-02:11:43.008326 139957265520192 EVENT_LOG_v1 {"time_micros": 1732385503008287, "cf_name": "default", "job": 39887, "event": "table_file_creation", "file_number": 288368, "file_size": 17895797, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4300085538, "largest_seqno": 4349869228, "table_properties": {"data_size": 15207744, "index_size": 812736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11792844, "raw_average_key_size": 28, "raw_value_size": 49698414, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 421173, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385501, "oldest_key_time": 0, "file_creation_time": 1732385501, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:11:43.027455 139957265520192 [compaction_job.cc:703] job-39887: subcompact[0], size: 0.554099 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39887/att-00 [288353,288354,288355,288356,288357,288358,288359,288360,288361,288362,288363,288364,288365,288366,288367,288368] 2024/11/24-02:11:43.122254 139957265520192 (Original Log Time 2024/11/24-02:10:53.016068) EVENT_LOG_v1 {"time_micros": 1732385453016009, "job": 39887, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [288316], "files_L3": [288280, 288281, 288282, 288283, 288284, 288285, 288286, 288287, 288288, 288289, 288250, 288251, 288252, 288253, 288254, 288347], "score": 0.669077, "input_data_size": 579816751, "oldest_snapshot_seqno": 4388892485} 2024/11/24-02:11:43.122258 139957265520192 (Original Log Time 2024/11/24-02:11:43.120982) [compaction_job.cc:2022] [default] [JOB 39887] Compacted 1@2 + 16@3 files to L3 => 554098627 bytes 2024/11/24-02:11:43.122259 139957265520192 (Original Log Time 2024/11/24-02:11:43.122189) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.6 rd, 11.1 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 525.9 +0.0 blob) out(528.4 +0.0 blob), read-write-amplify(39.9) write-amplify(19.5) OK, records in: 14446957, records dropped: 97956 output_compression: Snappy 2024/11/24-02:11:43.122265 139957265520192 (Original Log Time 2024/11/24-02:11:43.122232) EVENT_LOG_v1 {"time_micros": 1732385503122213, "cf": "default", "job": 39887, "event": "compaction_finished", "compaction_time_micros": 50011396, "compaction_time_cpu_micros": 21406126, "output_level": 3, "num_output_files": 16, "total_output_size": 554098627, "num_input_records": 14446957, "num_output_records": 14349001, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 378372893, "file_range_sync_nanos": 0, "file_fsync_nanos": 10900, "file_prepare_write_nanos": 38801, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/24-02:11:43.269336 139957265520192 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732385452,"executesMs":50240,"compactionJobId":39887,"attempt":0,"compactionInputRawBytes":2088667730,"compactionInputZipBytes":579816751,"compactionOutputRawBytes":0,"compactionOutputZipBytes":554098627,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-59b0775f43ddf60587f3a97401ba002a-c36ad0e0ef5aa2be-00"}}, response = 2024/11/24-02:11:43.269411 139957265520192 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39887/att-00: olev 3, work 50.240 s, result 7.673 ms, install 1.308 ms, input{raw 1.945 GiB zip 552.956 MiB}