2024/11/23-17:21:52.818511 139957584442944 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00: fork to child time = 0.022222 sec 2024/11/23-17:21:52.819263 139957584442944 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00 2024/11/23-17:21:52.827366 139957584442944 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00 2024/11/23-17:21:52.828581 139957584442944 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:21:52.828874 139957584442944 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:21:52.883158 139957584442944 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31804/att-00/MANIFEST-206347 2024/11/23-17:21:52.921800 139957584442944 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31804/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 272015, last_sequence is 4143781706, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271871 2024/11/23-17:21:52.921811 139957584442944 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:21:52.921989 139957584442944 [version_set.cc:5721] Creating manifest 272015 2024/11/23-17:21:52.958612 139957584442944 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:21:52.982452 139957584442944 [compaction_job.cc:2437] [default] [JOB 31804] Compacting 1@2 + 23@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:21:52.982481 139957584442944 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23806 Base level 2, inputs: [272006(27MB)], [271954(38MB) 271955(37MB) 271956(34MB) 271957(33MB) 271958(34MB) 271959(33MB) 271960(34MB) 271961(38MB) 271962(35MB) 271963(35MB) 271964(33MB) 271965(35MB) 271966(36MB) 271967(35MB) 271968(35MB) 271969(35MB) 271970(36MB) 271971(37MB) 271973(37MB) 271974(37MB) 271975(8119KB) 271921(36MB) 271922(37MB)] 2024/11/23-17:21:52.990629 139957584442944 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:21:52.990659 139957584442944 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:21:52.990678 139957584442944 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:21:52.990684 139957584442944 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:21:52.990693 139957584442944 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:21:53.634812 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst) /dev/shm/ToplingTemp/Topling-131194-G3EsUt 2024/11/23-17:21:54.228173 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst): first pass time = 0.59's, 226.214'MB/sec 2024/11/23-17:21:56.717763 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst): old prealloc_size = 147639500, real_size = 35645456 2024/11/23-17:21:56.718331 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929362 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.42 avg-zval = 36.19 2024/11/23-17:21:56.719162 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:21:57.551510 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272016.sst): second pass time = 2.49's, 43.475'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.962'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.298'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 311.363'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36688.647'MB/sec zip my value time = 2.49's, unzip length = 0.108'GB zip my value throughput = 43.475'MB/sec zip pipeline throughput = 68.160'MB/sec entries = 929362 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 36.19 usrkeys = 929362 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.42 avg-zval = 36.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929362 bytes = 3485120 } seq expand size = 6505534 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4466 value = 3.2169 dict = 1.00 all = 3.3565 } Zip/UnZip{ index = 0.0957 value = 0.3109 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 8.963321 GB avg = 0.168 KB (by entry num) total key len = 1.466843 GB avg = 0.028 KB total ukey len = 1.016465 GB avg = 0.015 KB total ukey num = 0.066479797 Billion total entry num = 0.053287526 Billion write speed all = 0.343037285 MB/sec (with seq num) write speed all = 0.329016713 MB/sec (without seq num) 2024/11/23-17:21:57.601891 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272016: 929362 keys, 39987236 bytes, temperature: kWarm 2024/11/23-17:21:57.602014 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353717601930, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272016, "file_size": 39987236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4143742235, "table_properties": {"data_size": 33633664, "index_size": 1779264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022136, "raw_average_key_size": 28, "raw_value_size": 108195652, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929362, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353713, "oldest_key_time": 0, "file_creation_time": 1732353713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272016, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:21:57.641778 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst) /dev/shm/ToplingTemp/Topling-131194-QCfrPr 2024/11/23-17:21:58.901949 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst): first pass time = 1.26's, 106.511'MB/sec 2024/11/23-17:22:00.715142 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst): old prealloc_size = 147639500, real_size = 36485552 2024/11/23-17:22:00.715728 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931965 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 116.02 avg-zval = 37.00 2024/11/23-17:22:00.716321 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:01.520495 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272017.sst): second pass time = 1.81's, 59.669'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.130'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.653'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.355'MB/sec write SST data time = 0.15's, 265.001'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29215.312'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.669'MB/sec zip pipeline throughput = 80.585'MB/sec entries = 931965 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.02 avg-zval = 37.00 usrkeys = 931965 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.32 avg-zval = 37.00 TagRS{ kinds = RS_Key0_TagN bytes = 145720 } TagArray{ size = 573412 bytes = 2078640 } seq expand size = 4013884 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5144 value = 3.1358 dict = 1.00 all = 3.3924 } Zip/UnZip{ index = 0.0951 value = 0.3189 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 9.075458 GB avg = 0.167 KB (by entry num) total key len = 1.492938 GB avg = 0.028 KB total ukey len = 1.035105 GB avg = 0.015 KB total ukey num = 0.067411762 Billion total entry num = 0.054219491 Billion write speed all = 0.347538214 MB/sec (with seq num) write speed all = 0.333274293 MB/sec (without seq num) 2024/11/23-17:22:01.575229 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272017: 931965 keys, 39564180 bytes, temperature: kWarm 2024/11/23-17:22:01.575324 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353721575274, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272017, "file_size": 39564180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34479648, "index_size": 1772736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095020, "raw_average_key_size": 28, "raw_value_size": 108122726, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931965, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353717, "oldest_key_time": 0, "file_creation_time": 1732353717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272017, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:01.651442 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst) /dev/shm/ToplingTemp/Topling-131194-r6jgSv 2024/11/23-17:22:02.591829 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst): first pass time = 0.94's, 142.736'MB/sec 2024/11/23-17:22:04.483049 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst): old prealloc_size = 147639500, real_size = 35329952 2024/11/23-17:22:04.483720 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081160 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0330 GB avg-val = 96.14 avg-zval = 30.51 2024/11/23-17:22:04.483902 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:05.178116 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272018.sst): second pass time = 1.89's, 54.984'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 52.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.764'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.489'MB/sec write SST data time = 0.14's, 258.131'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 33393.917'MB/sec zip my value time = 1.89's, unzip length = 0.104'GB zip my value throughput = 54.984'MB/sec zip pipeline throughput = 124.612'MB/sec entries = 1081160 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.14 avg-zval = 30.51 usrkeys = 1081160 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.47 avg-zval = 30.51 TagRS{ kinds = RS_Key0_TagN bytes = 169040 } TagArray{ size = 51310 bytes = 153952 } seq expand size = 359170 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0330 GB dict = 1.03 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4475 value = 3.1508 dict = 1.00 all = 3.6845 } Zip/UnZip{ index = 0.0957 value = 0.3174 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 9.179762 GB avg = 0.166 KB (by entry num) total key len = 1.523211 GB avg = 0.028 KB total ukey len = 1.056728 GB avg = 0.015 KB total ukey num = 0.068492922 Billion total entry num = 0.055300651 Billion write speed all = 0.351921403 MB/sec (with seq num) write speed all = 0.337374803 MB/sec (without seq num) 2024/11/23-17:22:05.222936 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272018: 1081160 keys, 36427922 bytes, temperature: kWarm 2024/11/23-17:22:05.223018 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353725222971, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272018, "file_size": 36427922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3283376628, "table_properties": {"data_size": 32989776, "index_size": 2069696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30272480, "raw_average_key_size": 28, "raw_value_size": 103945297, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081160, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353721, "oldest_key_time": 0, "file_creation_time": 1732353721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272018, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:05.258454 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst) /dev/shm/ToplingTemp/Topling-131194-cNkaRu 2024/11/23-17:22:06.335018 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst): first pass time = 1.08's, 124.677'MB/sec 2024/11/23-17:22:08.091048 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst): old prealloc_size = 147639500, real_size = 30526224 2024/11/23-17:22:08.091633 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928369 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.57 avg-zval = 30.72 2024/11/23-17:22:08.092474 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:08.915242 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272019.sst): second pass time = 1.76's, 61.658'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.296'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.478'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 269.842'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27122.739'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.658'MB/sec zip pipeline throughput = 131.799'MB/sec entries = 928369 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.57 avg-zval = 30.72 usrkeys = 928369 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.57 avg-zval = 30.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928369 bytes = 3481392 } seq expand size = 6498583 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4459 value = 3.7951 dict = 1.00 all = 3.8503 } Zip/UnZip{ index = 0.0957 value = 0.2635 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 9.294484 GB avg = 0.165 KB (by entry num) total key len = 1.549205 GB avg = 0.028 KB total ukey len = 1.075295 GB avg = 0.015 KB total ukey num = 0.069421291 Billion total entry num = 0.056229020 Billion write speed all = 0.356504450 MB/sec (with seq num) write speed all = 0.341715464 MB/sec (without seq num) 2024/11/23-17:22:08.964651 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272019: 928369 keys, 34859174 bytes, temperature: kWarm 2024/11/23-17:22:08.964747 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353728964694, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272019, "file_size": 34859174, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3283376629, "largest_seqno": 4012775885, "table_properties": {"data_size": 28516480, "index_size": 1777472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994332, "raw_average_key_size": 28, "raw_value_size": 108223434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928369, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353725, "oldest_key_time": 0, "file_creation_time": 1732353725, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272019, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:09.028683 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst) /dev/shm/ToplingTemp/Topling-131194-2pQTAr 2024/11/23-17:22:10.114813 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst): first pass time = 1.09's, 123.580'MB/sec 2024/11/23-17:22:11.929057 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst): old prealloc_size = 147639500, real_size = 31585168 2024/11/23-17:22:11.929575 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0296 GB avg-val = 116.22 avg-zval = 31.79 2024/11/23-17:22:11.930414 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:12.982075 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272020.sst): second pass time = 1.81's, 59.674'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 34.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3573.683'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.17's, 205.282'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29291.800'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.674'MB/sec zip pipeline throughput = 120.327'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 31.79 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5121 value = 3.6563 dict = 1.00 all = 3.7480 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2668 } ---------------------------- total value len = 9.409158 GB avg = 0.165 KB (by entry num) total key len = 1.575263 GB avg = 0.028 KB total ukey len = 1.093908 GB avg = 0.016 KB total ukey num = 0.070351937 Billion total entry num = 0.057159666 Billion write speed all = 0.361082981 MB/sec (with seq num) write speed all = 0.346051233 MB/sec (without seq num) 2024/11/23-17:22:13.028310 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272020: 930646 keys, 35810034 bytes, temperature: kWarm 2024/11/23-17:22:13.028379 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353733028343, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272020, "file_size": 35810034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 29581696, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353729, "oldest_key_time": 0, "file_creation_time": 1732353729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272020, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:13.066219 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst) /dev/shm/ToplingTemp/Topling-131194-Dtk63s 2024/11/23-17:22:13.991900 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst): first pass time = 0.93's, 144.999'MB/sec 2024/11/23-17:22:15.490175 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst): old prealloc_size = 147639500, real_size = 31580624 2024/11/23-17:22:15.490729 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 115.97 avg-zval = 31.71 2024/11/23-17:22:15.491547 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:16.522630 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272021.sst): second pass time = 1.50's, 72.198'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 34.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.455'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.13's, 279.721'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35997.796'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.198'MB/sec zip pipeline throughput = 72.198'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.97 avg-zval = 31.71 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.97 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4444 value = 3.6572 dict = 1.00 all = 3.7602 } Zip/UnZip{ index = 0.0957 value = 0.2734 dict = 1.00 all = 0.2659 } ---------------------------- total value len = 9.523799 GB avg = 0.164 KB (by entry num) total key len = 1.601366 GB avg = 0.028 KB total ukey len = 1.112553 GB avg = 0.016 KB total ukey num = 0.071284177 Billion total entry num = 0.058091906 Billion write speed all = 0.365666981 MB/sec (with seq num) write speed all = 0.350391852 MB/sec (without seq num) 2024/11/23-17:22:16.569055 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272021: 932240 keys, 35694146 bytes, temperature: kWarm 2024/11/23-17:22:16.569126 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353736569090, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272021, "file_size": 35694146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 29562240, "index_size": 1785152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353733, "oldest_key_time": 0, "file_creation_time": 1732353733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272021, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:16.607603 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst) /dev/shm/ToplingTemp/Topling-131194-S9hPys 2024/11/23-17:22:17.516510 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst): first pass time = 0.91's, 147.678'MB/sec 2024/11/23-17:22:19.087395 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst): old prealloc_size = 147639500, real_size = 32285600 2024/11/23-17:22:19.087932 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925338 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 117.05 avg-zval = 32.74 2024/11/23-17:22:19.088746 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:20.115539 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272022.sst): second pass time = 1.57's, 69.012'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 35.609'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3513.078'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 287.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33069.417'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.012'MB/sec zip pipeline throughput = 69.012'MB/sec entries = 925338 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 32.74 usrkeys = 925338 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925338 bytes = 3470032 } seq expand size = 6477366 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5137 value = 3.5753 dict = 1.00 all = 3.6661 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 9.638585 GB avg = 0.163 KB (by entry num) total key len = 1.627275 GB avg = 0.028 KB total ukey len = 1.131060 GB avg = 0.016 KB total ukey num = 0.072209515 Billion total entry num = 0.059017244 Billion write speed all = 0.370247690 MB/sec (with seq num) write speed all = 0.354731077 MB/sec (without seq num) 2024/11/23-17:22:20.173044 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272022: 925338 keys, 36610572 bytes, temperature: kWarm 2024/11/23-17:22:20.173119 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353740173079, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272022, "file_size": 36610572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775935, "table_properties": {"data_size": 30293840, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909464, "raw_average_key_size": 28, "raw_value_size": 108308324, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353736, "oldest_key_time": 0, "file_creation_time": 1732353736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272022, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:20.225620 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst) /dev/shm/ToplingTemp/Topling-131194-EJX3sv 2024/11/23-17:22:21.165499 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst): first pass time = 0.94's, 142.809'MB/sec 2024/11/23-17:22:22.732721 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst): old prealloc_size = 147639500, real_size = 35772768 2024/11/23-17:22:22.733302 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927163 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 116.76 avg-zval = 36.42 2024/11/23-17:22:22.734093 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:23.765601 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272023.sst): second pass time = 1.57's, 69.139'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.716'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.153'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 303.542'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33523.224'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.139'MB/sec zip pipeline throughput = 69.139'MB/sec entries = 927163 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.76 avg-zval = 36.42 usrkeys = 927163 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.76 avg-zval = 36.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927163 bytes = 3360976 } seq expand size = 6490141 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4437 value = 3.2062 dict = 1.00 all = 3.3564 } Zip/UnZip{ index = 0.0958 value = 0.3119 dict = 1.00 all = 0.2979 } ---------------------------- total value len = 9.753332 GB avg = 0.163 KB (by entry num) total key len = 1.653236 GB avg = 0.028 KB total ukey len = 1.149603 GB avg = 0.016 KB total ukey num = 0.073136678 Billion total entry num = 0.059944407 Billion write speed all = 0.374827033 MB/sec (with seq num) write speed all = 0.359068545 MB/sec (without seq num) 2024/11/23-17:22:23.815608 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272023: 927163 keys, 39988212 bytes, temperature: kWarm 2024/11/23-17:22:23.815677 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353743815643, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272023, "file_size": 39988212, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33765264, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960564, "raw_average_key_size": 28, "raw_value_size": 108257254, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353740, "oldest_key_time": 0, "file_creation_time": 1732353740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272023, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:23.856508 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst) /dev/shm/ToplingTemp/Topling-131194-kQdRrv 2024/11/23-17:22:24.924258 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst): first pass time = 1.07's, 125.706'MB/sec 2024/11/23-17:22:26.381753 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst): old prealloc_size = 147639500, real_size = 32374048 2024/11/23-17:22:26.382454 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927695 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.68 avg-zval = 32.74 2024/11/23-17:22:26.383941 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:27.404336 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272024.sst): second pass time = 1.46's, 74.307'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 35.897'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2598.524'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.15's, 236.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34248.669'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.307'MB/sec zip pipeline throughput = 74.307'MB/sec entries = 927695 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 32.74 usrkeys = 927695 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927695 bytes = 3478864 } seq expand size = 6493865 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5145 value = 3.5633 dict = 1.00 all = 3.6564 } Zip/UnZip{ index = 0.0951 value = 0.2806 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 9.868068 GB avg = 0.162 KB (by entry num) total key len = 1.679211 GB avg = 0.028 KB total ukey len = 1.168157 GB avg = 0.016 KB total ukey num = 0.074064373 Billion total entry num = 0.060872102 Billion write speed all = 0.379405542 MB/sec (with seq num) write speed all = 0.363405089 MB/sec (without seq num) 2024/11/23-17:22:27.451101 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272024: 927695 keys, 36707172 bytes, temperature: kWarm 2024/11/23-17:22:27.451173 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353747451134, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272024, "file_size": 36707172, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30377360, "index_size": 1764608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975460, "raw_average_key_size": 28, "raw_value_size": 108242342, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927695, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353743, "oldest_key_time": 0, "file_creation_time": 1732353743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272024, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:27.489546 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst) /dev/shm/ToplingTemp/Topling-131194-4JruCv 2024/11/23-17:22:28.783500 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst): first pass time = 1.29's, 103.730'MB/sec 2024/11/23-17:22:30.548103 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst): old prealloc_size = 147639500, real_size = 33988432 2024/11/23-17:22:30.548669 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928959 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.48 avg-zval = 34.42 2024/11/23-17:22:30.549373 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:31.368005 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272025.sst): second pass time = 1.76's, 61.363'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.090'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.555'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 295.059'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 39978.739'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.363'MB/sec zip pipeline throughput = 99.900'MB/sec entries = 928959 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.48 avg-zval = 34.42 usrkeys = 928959 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.10 avg-zval = 34.42 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 745962 bytes = 2797376 } seq expand size = 5221734 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4458 value = 3.3839 dict = 1.00 all = 3.5519 } Zip/UnZip{ index = 0.0957 value = 0.2955 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 9.981497 GB avg = 0.162 KB (by entry num) total key len = 1.705222 GB avg = 0.028 KB total ukey len = 1.186736 GB avg = 0.016 KB total ukey num = 0.074993332 Billion total entry num = 0.061801061 Billion write speed all = 0.383937064 MB/sec (with seq num) write speed all = 0.367694547 MB/sec (without seq num) 2024/11/23-17:22:31.416508 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272025: 928959 keys, 37787842 bytes, temperature: kWarm 2024/11/23-17:22:31.416578 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353751416542, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272025, "file_size": 37787842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 31977408, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26010852, "raw_average_key_size": 28, "raw_value_size": 108206958, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928959, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353747, "oldest_key_time": 0, "file_creation_time": 1732353747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272025, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:31.456240 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst) /dev/shm/ToplingTemp/Topling-131194-VXC2Bu 2024/11/23-17:22:32.318913 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst): first pass time = 0.86's, 155.591'MB/sec 2024/11/23-17:22:33.642312 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst): old prealloc_size = 147639500, real_size = 31206016 2024/11/23-17:22:33.642854 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 115.79 avg-zval = 31.27 2024/11/23-17:22:33.643643 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:34.482492 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272026.sst): second pass time = 1.32's, 81.753'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3433.265'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 277.097'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36149.224'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.753'MB/sec zip pipeline throughput = 81.753'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.79 avg-zval = 31.27 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.79 avg-zval = 31.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4449 value = 3.7033 dict = 1.00 all = 3.7872 } Zip/UnZip{ index = 0.0957 value = 0.2700 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 10.096112 GB avg = 0.161 KB (by entry num) total key len = 1.731359 GB avg = 0.028 KB total ukey len = 1.205405 GB avg = 0.016 KB total ukey num = 0.075926789 Billion total entry num = 0.062734518 Billion write speed all = 0.388521357 MB/sec (with seq num) write speed all = 0.372035196 MB/sec (without seq num) 2024/11/23-17:22:34.526512 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272026: 933457 keys, 35440262 bytes, temperature: kWarm 2024/11/23-17:22:34.526579 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353754526545, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272026, "file_size": 35440262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29185072, "index_size": 1787392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353751, "oldest_key_time": 0, "file_creation_time": 1732353751, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272026, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:34.566123 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst) /dev/shm/ToplingTemp/Topling-131194-g0DYou 2024/11/23-17:22:36.058113 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst): first pass time = 1.49's, 89.961'MB/sec 2024/11/23-17:22:37.423787 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst): old prealloc_size = 147639500, real_size = 33046832 2024/11/23-17:22:37.424346 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0310 GB avg-val = 116.16 avg-zval = 33.33 2024/11/23-17:22:37.425202 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:38.314356 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272027.sst): second pass time = 1.36's, 79.276'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.855'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.615'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.14's, 276.062'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33553.385'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.276'MB/sec zip pipeline throughput = 79.276'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 33.33 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4458 value = 3.4852 dict = 1.00 all = 3.6009 } Zip/UnZip{ index = 0.0957 value = 0.2869 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 10.210778 GB avg = 0.160 KB (by entry num) total key len = 1.757428 GB avg = 0.028 KB total ukey len = 1.224026 GB avg = 0.016 KB total ukey num = 0.076857822 Billion total entry num = 0.063665551 Billion write speed all = 0.393094893 MB/sec (with seq num) write speed all = 0.376366169 MB/sec (without seq num) 2024/11/23-17:22:38.361992 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272027: 931033 keys, 37273436 bytes, temperature: kWarm 2024/11/23-17:22:38.362064 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353758362027, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272027, "file_size": 37273436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31031328, "index_size": 1782592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353754, "oldest_key_time": 0, "file_creation_time": 1732353754, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272027, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:38.400873 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst) /dev/shm/ToplingTemp/Topling-131194-Z152Qt 2024/11/23-17:22:39.448542 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst): first pass time = 1.05's, 128.115'MB/sec 2024/11/23-17:22:41.357268 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst): old prealloc_size = 147639500, real_size = 34466880 2024/11/23-17:22:41.357871 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923638 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.31 avg-zval = 35.15 2024/11/23-17:22:41.358717 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:42.211101 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272028.sst): second pass time = 1.91's, 56.801'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3051.667'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 301.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34868.960'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.801'MB/sec zip pipeline throughput = 56.801'MB/sec entries = 923638 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.31 avg-zval = 35.15 usrkeys = 923638 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.31 avg-zval = 35.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923638 bytes = 3463664 } seq expand size = 6465466 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4420 value = 3.3374 dict = 1.00 all = 3.4603 } Zip/UnZip{ index = 0.0958 value = 0.2996 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 10.325600 GB avg = 0.160 KB (by entry num) total key len = 1.783290 GB avg = 0.028 KB total ukey len = 1.242499 GB avg = 0.016 KB total ukey num = 0.077781460 Billion total entry num = 0.064589189 Billion write speed all = 0.397664727 MB/sec (with seq num) write speed all = 0.380695480 MB/sec (without seq num) 2024/11/23-17:22:42.259468 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272028: 923638 keys, 38787896 bytes, temperature: kWarm 2024/11/23-17:22:42.259537 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353762259500, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272028, "file_size": 38787896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32466672, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25861864, "raw_average_key_size": 28, "raw_value_size": 108355884, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353758, "oldest_key_time": 0, "file_creation_time": 1732353758, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272028, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:42.295606 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst) /dev/shm/ToplingTemp/Topling-131194-bjQ8dv 2024/11/23-17:22:43.652129 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst): first pass time = 1.36's, 98.945'MB/sec 2024/11/23-17:22:44.984009 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst): old prealloc_size = 147639500, real_size = 32947072 2024/11/23-17:22:44.984552 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 115.73 avg-zval = 33.13 2024/11/23-17:22:44.985348 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:45.844129 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272029.sst): second pass time = 1.33's, 81.191'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.180'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.681'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34174.021'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.191'MB/sec zip pipeline throughput = 81.191'MB/sec entries = 933794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 33.13 usrkeys = 933794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933794 bytes = 3385024 } seq expand size = 6536558 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5142 value = 3.4933 dict = 1.00 all = 3.6097 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 10.440208 GB avg = 0.159 KB (by entry num) total key len = 1.809436 GB avg = 0.028 KB total ukey len = 1.261174 GB avg = 0.016 KB total ukey num = 0.078715254 Billion total entry num = 0.065522983 Billion write speed all = 0.402239213 MB/sec (with seq num) write speed all = 0.385026688 MB/sec (without seq num) 2024/11/23-17:22:45.892669 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272029: 933794 keys, 37182714 bytes, temperature: kWarm 2024/11/23-17:22:45.892740 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353765892703, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272029, "file_size": 37182714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30937200, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146232, "raw_average_key_size": 28, "raw_value_size": 108071628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353762, "oldest_key_time": 0, "file_creation_time": 1732353762, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272029, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:45.937827 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst) /dev/shm/ToplingTemp/Topling-131194-zYeAAu 2024/11/23-17:22:47.021282 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst): first pass time = 1.08's, 123.884'MB/sec 2024/11/23-17:22:48.363267 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst): old prealloc_size = 147639500, real_size = 33370560 2024/11/23-17:22:48.363814 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924147 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.23 avg-zval = 33.94 2024/11/23-17:22:48.364626 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:49.207905 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272030.sst): second pass time = 1.34's, 80.780'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.923'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 298.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34743.707'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.780'MB/sec zip pipeline throughput = 80.780'MB/sec entries = 924147 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.23 avg-zval = 33.94 usrkeys = 924147 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.23 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924147 bytes = 3465568 } seq expand size = 6469029 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4436 value = 3.4537 dict = 1.00 all = 3.5608 } Zip/UnZip{ index = 0.0958 value = 0.2895 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 10.555019 GB avg = 0.159 KB (by entry num) total key len = 1.835312 GB avg = 0.028 KB total ukey len = 1.279657 GB avg = 0.016 KB total ukey num = 0.079639401 Billion total entry num = 0.066447130 Billion write speed all = 0.406813985 MB/sec (with seq num) write speed all = 0.389360620 MB/sec (without seq num) 2024/11/23-17:22:49.254687 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272030: 924147 keys, 37692954 bytes, temperature: kWarm 2024/11/23-17:22:49.254756 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353769254719, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272030, "file_size": 37692954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31369520, "index_size": 1769792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876116, "raw_average_key_size": 28, "raw_value_size": 108341714, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924147, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353765, "oldest_key_time": 0, "file_creation_time": 1732353765, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272030, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:49.292843 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst) /dev/shm/ToplingTemp/Topling-131194-OWcYxs 2024/11/23-17:22:50.309435 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst): first pass time = 1.02's, 132.033'MB/sec 2024/11/23-17:22:51.622381 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst): old prealloc_size = 147639500, real_size = 33073088 2024/11/23-17:22:51.622967 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927082 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.77 avg-zval = 33.50 2024/11/23-17:22:51.625491 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:52.478195 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272031.sst): second pass time = 1.31's, 82.534'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.707'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.433'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 295.036'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30671.588'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.534'MB/sec zip pipeline throughput = 82.534'MB/sec entries = 927082 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.77 avg-zval = 33.50 usrkeys = 927082 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.77 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927082 bytes = 3476576 } seq expand size = 6489574 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3758 value = 3.4862 dict = 1.00 all = 3.5883 } Zip/UnZip{ index = 0.0964 value = 0.2868 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 10.669768 GB avg = 0.158 KB (by entry num) total key len = 1.861270 GB avg = 0.028 KB total ukey len = 1.298199 GB avg = 0.016 KB total ukey num = 0.080566483 Billion total entry num = 0.067374212 Billion write speed all = 0.411389685 MB/sec (with seq num) write speed all = 0.393694707 MB/sec (without seq num) 2024/11/23-17:22:52.524616 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272031: 927082 keys, 37403920 bytes, temperature: kWarm 2024/11/23-17:22:52.524688 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353772524651, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272031, "file_size": 37403920, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31054128, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958296, "raw_average_key_size": 28, "raw_value_size": 108259552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927082, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353769, "oldest_key_time": 0, "file_creation_time": 1732353769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272031, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:52.560288 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst) /dev/shm/ToplingTemp/Topling-131194-RVkDNu 2024/11/23-17:22:53.394841 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst): first pass time = 0.83's, 160.835'MB/sec 2024/11/23-17:22:54.847857 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst): old prealloc_size = 147639500, real_size = 35530096 2024/11/23-17:22:54.848463 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.33 avg-zval = 36.06 2024/11/23-17:22:54.848825 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:55.597515 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272032.sst): second pass time = 1.45's, 74.493'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 50.245'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3027.842'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.372'MB/sec write SST data time = 0.12's, 304.283'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32644.746'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.493'MB/sec zip pipeline throughput = 74.493'MB/sec entries = 929905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.33 avg-zval = 36.06 usrkeys = 929905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.96 avg-zval = 36.06 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 348415 bytes = 1132368 } seq expand size = 2438905 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5117 value = 3.2266 dict = 1.00 all = 3.5635 } Zip/UnZip{ index = 0.0951 value = 0.3099 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 10.780387 GB avg = 0.158 KB (by entry num) total key len = 1.887308 GB avg = 0.028 KB total ukey len = 1.316797 GB avg = 0.016 KB total ukey num = 0.081496388 Billion total entry num = 0.068304117 Billion write speed all = 0.415833493 MB/sec (with seq num) write speed all = 0.397896124 MB/sec (without seq num) 2024/11/23-17:22:55.643736 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272032: 929905 keys, 37665105 bytes, temperature: kWarm 2024/11/23-17:22:55.643807 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353775643770, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272032, "file_size": 37665105, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33528160, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26037340, "raw_average_key_size": 28, "raw_value_size": 108180426, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353772, "oldest_key_time": 0, "file_creation_time": 1732353772, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272032, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:55.681934 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst) /dev/shm/ToplingTemp/Topling-131194-e7icis 2024/11/23-17:22:56.679444 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst): first pass time = 1.00's, 134.558'MB/sec 2024/11/23-17:22:58.165737 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst): old prealloc_size = 147639500, real_size = 37174384 2024/11/23-17:22:58.166347 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926207 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.91 avg-zval = 37.97 2024/11/23-17:22:58.166648 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:22:58.880690 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272033.sst): second pass time = 1.49's, 72.918'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.482'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3015.942'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.392'MB/sec write SST data time = 0.12's, 315.618'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33915.939'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.918'MB/sec zip pipeline throughput = 72.918'MB/sec entries = 926207 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 37.97 usrkeys = 926207 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.47 avg-zval = 37.97 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 206104 bytes = 772912 } seq expand size = 1442728 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4468 value = 3.0789 dict = 1.00 all = 3.4458 } Zip/UnZip{ index = 0.0957 value = 0.3248 dict = 1.00 all = 0.2902 } ---------------------------- total value len = 10.890114 GB avg = 0.157 KB (by entry num) total key len = 1.913242 GB avg = 0.028 KB total ukey len = 1.335321 GB avg = 0.016 KB total ukey num = 0.082422595 Billion total entry num = 0.069230324 Billion write speed all = 0.420241435 MB/sec (with seq num) write speed all = 0.402062794 MB/sec (without seq num) 2024/11/23-17:22:58.929039 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272033: 926207 keys, 38950589 bytes, temperature: kWarm 2024/11/23-17:22:58.929108 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353778929072, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272033, "file_size": 38950589, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4022858715, "table_properties": {"data_size": 35169440, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933796, "raw_average_key_size": 28, "raw_value_size": 108284038, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926207, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353775, "oldest_key_time": 0, "file_creation_time": 1732353775, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272033, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:22:58.968741 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst) /dev/shm/ToplingTemp/Topling-131194-YYDUYr 2024/11/23-17:22:59.814055 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst): first pass time = 0.85's, 158.788'MB/sec 2024/11/23-17:23:01.151204 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst): old prealloc_size = 147639500, real_size = 35485024 2024/11/23-17:23:01.151754 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927304 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.74 avg-zval = 36.11 2024/11/23-17:23:01.152525 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:23:01.980213 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272034.sst): second pass time = 1.34's, 81.010'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.611'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3355.006'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 314.379'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33288.249'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.010'MB/sec zip pipeline throughput = 81.010'MB/sec entries = 927304 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 36.11 usrkeys = 927304 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 36.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927304 bytes = 3129664 } seq expand size = 6491128 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5135 value = 3.2325 dict = 1.00 all = 3.4006 } Zip/UnZip{ index = 0.0951 value = 0.3094 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 11.004858 GB avg = 0.157 KB (by entry num) total key len = 1.939206 GB avg = 0.028 KB total ukey len = 1.353867 GB avg = 0.016 KB total ukey num = 0.083349899 Billion total entry num = 0.070157628 Billion write speed all = 0.424816671 MB/sec (with seq num) write speed all = 0.406396411 MB/sec (without seq num) 2024/11/23-17:23:02.027639 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272034: 927304 keys, 39469158 bytes, temperature: kWarm 2024/11/23-17:23:02.027714 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353782027676, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272034, "file_size": 39469158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4022858716, "largest_seqno": 4143742121, "table_properties": {"data_size": 33488976, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964512, "raw_average_key_size": 28, "raw_value_size": 108253280, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927304, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353778, "oldest_key_time": 0, "file_creation_time": 1732353778, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272034, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:23:02.067514 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst) /dev/shm/ToplingTemp/Topling-131194-KrrMcs 2024/11/23-17:23:03.079581 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst): first pass time = 1.01's, 132.626'MB/sec 2024/11/23-17:23:04.380410 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst): old prealloc_size = 147639500, real_size = 35086528 2024/11/23-17:23:04.380940 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928850 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.50 avg-zval = 35.62 2024/11/23-17:23:04.381708 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:23:05.211630 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272035.sst): second pass time = 1.30's, 83.242'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.870'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3553.253'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.12's, 315.906'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31831.431'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.242'MB/sec zip pipeline throughput = 83.242'MB/sec entries = 928850 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 35.62 usrkeys = 928850 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 35.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928850 bytes = 3018784 } seq expand size = 6501950 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5146 value = 3.2704 dict = 1.00 all = 3.4452 } Zip/UnZip{ index = 0.0951 value = 0.3058 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 11.119570 GB avg = 0.156 KB (by entry num) total key len = 1.965214 GB avg = 0.028 KB total ukey len = 1.372444 GB avg = 0.016 KB total ukey num = 0.084278749 Billion total entry num = 0.071086478 Billion write speed all = 0.429389475 MB/sec (with seq num) write speed all = 0.410727319 MB/sec (without seq num) 2024/11/23-17:23:05.259311 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272035: 928850 keys, 38958300 bytes, temperature: kWarm 2024/11/23-17:23:05.259380 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353785259344, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272035, "file_size": 38958300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4143741639, "table_properties": {"data_size": 33087344, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007800, "raw_average_key_size": 28, "raw_value_size": 108210056, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928850, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353782, "oldest_key_time": 0, "file_creation_time": 1732353782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272035, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:23:05.306391 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst) /dev/shm/ToplingTemp/Topling-131194-QvFjDt 2024/11/23-17:23:06.262114 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst): first pass time = 0.96's, 140.441'MB/sec 2024/11/23-17:23:07.566686 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst): old prealloc_size = 147639500, real_size = 33598208 2024/11/23-17:23:07.567248 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927343 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.73 avg-zval = 34.08 2024/11/23-17:23:07.568014 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:23:08.410819 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272036.sst): second pass time = 1.30's, 83.029'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.595'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 301.602'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35962.656'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.029'MB/sec zip pipeline throughput = 83.029'MB/sec entries = 927343 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 34.08 usrkeys = 927343 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 34.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927343 bytes = 3129792 } seq expand size = 6491401 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5124 value = 3.4255 dict = 1.00 all = 3.5709 } Zip/UnZip{ index = 0.0951 value = 0.2919 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 11.234314 GB avg = 0.156 KB (by entry num) total key len = 1.991179 GB avg = 0.028 KB total ukey len = 1.390991 GB avg = 0.016 KB total ukey num = 0.085206092 Billion total entry num = 0.072013821 Billion write speed all = 0.433961419 MB/sec (with seq num) write speed all = 0.415057794 MB/sec (without seq num) 2024/11/23-17:23:08.456308 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272036: 927343 keys, 37586434 bytes, temperature: kWarm 2024/11/23-17:23:08.456380 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353788456343, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272036, "file_size": 37586434, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4143742211, "table_properties": {"data_size": 31601904, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965604, "raw_average_key_size": 28, "raw_value_size": 108252138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927343, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353785, "oldest_key_time": 0, "file_creation_time": 1732353785, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272036, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:23:08.493045 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst) /dev/shm/ToplingTemp/Topling-131194-V2TGVs 2024/11/23-17:23:09.535959 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst): first pass time = 1.04's, 128.699'MB/sec 2024/11/23-17:23:10.863794 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst): old prealloc_size = 147639500, real_size = 34361024 2024/11/23-17:23:10.864351 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929428 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.41 avg-zval = 34.82 2024/11/23-17:23:10.865660 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:23:11.704929 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272037.sst): second pass time = 1.33's, 81.531'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.457'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.587'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 302.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32177.550'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.531'MB/sec zip pipeline throughput = 81.531'MB/sec entries = 929428 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 34.82 usrkeys = 929428 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 34.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929428 bytes = 3020656 } seq expand size = 6505996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5131 value = 3.3434 dict = 1.00 all = 3.5104 } Zip/UnZip{ index = 0.0951 value = 0.2991 dict = 1.00 all = 0.2849 } ---------------------------- total value len = 11.349014 GB avg = 0.156 KB (by entry num) total key len = 2.017203 GB avg = 0.028 KB total ukey len = 1.409580 GB avg = 0.016 KB total ukey num = 0.086135520 Billion total entry num = 0.072943249 Billion write speed all = 0.438531518 MB/sec (with seq num) write speed all = 0.419385988 MB/sec (without seq num) 2024/11/23-17:23:11.751775 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272037: 929428 keys, 38233840 bytes, temperature: kWarm 2024/11/23-17:23:11.751846 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353791751810, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272037, "file_size": 38233840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4143742143, "table_properties": {"data_size": 32360368, "index_size": 1768128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023984, "raw_average_key_size": 28, "raw_value_size": 108193824, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353788, "oldest_key_time": 0, "file_creation_time": 1732353788, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272037, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:23:11.789219 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst) /dev/shm/ToplingTemp/Topling-131194-HMcAns 2024/11/23-17:23:11.989355 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst): first pass time = 0.20's, 245.567'MB/sec 2024/11/23-17:23:12.463603 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst): old prealloc_size = 147639500, real_size = 13574384 2024/11/23-17:23:12.463839 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 340448 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0396 GB zip-val = 0.0128 GB avg-val = 116.33 avg-zval = 37.72 2024/11/23-17:23:12.464179 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.049 GB 2024/11/23-17:23:12.765103 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00/272038.sst): second pass time = 0.47's, 83.640'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.30's, 49.765'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.630'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.334'MB/sec write SST data time = 0.05's, 323.076'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.405'MB, throughput = 14013.855'MB/sec zip my value time = 0.47's, unzip length = 0.040'GB zip my value throughput = 83.640'MB/sec zip pipeline throughput = 83.640'MB/sec entries = 340448 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.33 avg-zval = 37.72 usrkeys = 340448 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.33 avg-zval = 37.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 340448 bytes = 1106464 } seq expand size = 2383136 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0068 GB value = 0.0396 GB dict = 0.40 MB all = 0.0491 GB } __ZipSize{ index = 0.0006 GB value = 0.0128 GB dict = 0.40 MB all = 0.0150 GB } UnZip/Zip{ index = 10.4973 value = 3.0843 dict = 1.00 all = 3.2751 } Zip/UnZip{ index = 0.0953 value = 0.3242 dict = 1.00 all = 0.3053 } ---------------------------- total value len = 11.391001 GB avg = 0.155 KB (by entry num) total key len = 2.026736 GB avg = 0.028 KB total ukey len = 1.416389 GB avg = 0.016 KB total ukey num = 0.086475968 Billion total entry num = 0.073283697 Billion write speed all = 0.440206525 MB/sec (with seq num) write speed all = 0.420972306 MB/sec (without seq num) 2024/11/23-17:23:12.787832 139957584442944 [compaction_job.cc:1948] [default] [JOB 31804] Generated table #272038: 340448 keys, 15003251 bytes, temperature: kWarm 2024/11/23-17:23:12.787899 139957584442944 EVENT_LOG_v1 {"time_micros": 1732353792787865, "cf_name": "default", "job": 31804, "event": "table_file_creation", "file_number": 272038, "file_size": 15003251, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4083442763, "largest_seqno": 4142229803, "table_properties": {"data_size": 12840480, "index_size": 648640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9532544, "raw_average_key_size": 28, "raw_value_size": 39604368, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 340448, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353791, "oldest_key_time": 0, "file_creation_time": 1732353791, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:23:12.803014 139957584442944 [compaction_job.cc:703] job-31804: subcompact[0], size: 0.843084 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31804/att-00 [272016,272017,272018,272019,272020,272021,272022,272023,272024,272025,272026,272027,272028,272029,272030,272031,272032,272033,272034,272035,272036,272037,272038] 2024/11/23-17:23:12.970688 139957584442944 (Original Log Time 2024/11/23-17:21:52.982555) EVENT_LOG_v1 {"time_micros": 1732353712982500, "job": 31804, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [272006], "files_L3": [271954, 271955, 271956, 271957, 271958, 271959, 271960, 271961, 271962, 271963, 271964, 271965, 271966, 271967, 271968, 271969, 271970, 271971, 271973, 271974, 271975, 271921, 271922], "score": 0.997929, "input_data_size": 866439494, "oldest_snapshot_seqno": 4143781647} 2024/11/23-17:23:12.970692 139957584442944 (Original Log Time 2024/11/23-17:23:12.969088) [compaction_job.cc:2022] [default] [JOB 31804] Compacted 1@2 + 23@3 files to L3 => 843084349 bytes 2024/11/23-17:23:12.970693 139957584442944 (Original Log Time 2024/11/23-17:23:12.970613) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 10.9 rd, 10.6 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 799.2 +0.0 blob) out(804.0 +0.0 blob), read-write-amplify(60.1) write-amplify(29.6) OK, records in: 21000659, records dropped: 75126 output_compression: Snappy 2024/11/23-17:23:12.970696 139957584442944 (Original Log Time 2024/11/23-17:23:12.970663) EVENT_LOG_v1 {"time_micros": 1732353792970641, "cf": "default", "job": 31804, "event": "compaction_finished", "compaction_time_micros": 79820466, "compaction_time_cpu_micros": 33193340, "output_level": 3, "num_output_files": 23, "total_output_size": 843084349, "num_input_records": 21000659, "num_output_records": 20925533, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 595524837, "file_range_sync_nanos": 0, "file_fsync_nanos": 17356, "file_prepare_write_nanos": 64668, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:23:13.118896 139957584442944 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732353712,"executesMs":80142,"compactionJobId":31804,"attempt":0,"compactionInputRawBytes":3012596423,"compactionInputZipBytes":866439494,"compactionOutputRawBytes":0,"compactionOutputZipBytes":843084349,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5c91b7204b2c38a1a23e5e6a0a230a0f-6fe0e5fca2cd25c2-00"}}, response = 2024/11/23-17:23:13.119020 139957584442944 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31804/att-00: olev 3, work 80.143 s, result 7.774 ms, install 1.623 ms, input{raw 2.806 GiB zip 826.301 MiB}