2024/11/23-18:20:27.555662 139957592835648 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00: fork to child time = 0.014547 sec 2024/11/23-18:20:27.556335 139957592835648 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00 2024/11/23-18:20:27.564933 139957592835648 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00 2024/11/23-18:20:27.565937 139957592835648 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:20:27.566154 139957592835648 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:20:27.614618 139957592835648 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32969/att-00/MANIFEST-206347 2024/11/23-18:20:27.631056 139957592835648 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32969/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 274518, last_sequence is 4178803338, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 274356 2024/11/23-18:20:27.631062 139957592835648 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:20:27.631190 139957592835648 [version_set.cc:5721] Creating manifest 274518 2024/11/23-18:20:27.666001 139957592835648 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:20:27.688237 139957592835648 [compaction_job.cc:2437] [default] [JOB 32969] Compacting 1@2 + 19@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-18:20:27.688253 139957592835648 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24678 Base level 2, inputs: [274486(27MB)], [274402(36MB) 274403(36MB) 274404(36MB) 274443(38MB) 274444(37MB) 274445(35MB) 274446(38MB) 274447(35MB) 274448(34MB) 274449(34MB) 274450(35MB) 274451(35MB) 274452(36MB) 274453(34MB) 274454(35MB) 274455(36MB) 274456(37MB) 274457(37MB) 274511(37MB)] 2024/11/23-18:20:27.696536 139957592835648 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:20:27.696561 139957592835648 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:20:27.696576 139957592835648 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:20:27.696581 139957592835648 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:20:27.696592 139957592835648 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:20:28.241701 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst) /dev/shm/ToplingTemp/Topling-147862-LXCn5N 2024/11/23-18:20:29.261264 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst): first pass time = 1.02's, 131.649'MB/sec 2024/11/23-18:20:31.408850 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst): old prealloc_size = 147639500, real_size = 34720608 2024/11/23-18:20:31.409447 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925336 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.05 avg-zval = 35.37 2024/11/23-18:20:31.410246 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:32.279702 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274519.sst): second pass time = 2.15's, 50.468'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.306'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3038.612'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.14's, 285.660'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35596.021'MB/sec zip my value time = 2.15's, unzip length = 0.108'GB zip my value throughput = 50.468'MB/sec zip pipeline throughput = 90.163'MB/sec entries = 925336 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 35.37 usrkeys = 925336 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 35.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925336 bytes = 3007360 } seq expand size = 6477352 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5137 value = 3.3093 dict = 1.00 all = 3.4787 } Zip/UnZip{ index = 0.0951 value = 0.3022 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 8.963406 GB avg = 0.168 KB (by entry num) total key len = 1.466730 GB avg = 0.028 KB total ukey len = 1.016385 GB avg = 0.015 KB total ukey num = 0.066475771 Billion total entry num = 0.053283500 Billion write speed all = 0.307491627 MB/sec (with seq num) write speed all = 0.294924788 MB/sec (without seq num) 2024/11/23-18:20:32.327138 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274519: 925336 keys, 38583104 bytes, temperature: kWarm 2024/11/23-18:20:32.327244 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357232327180, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274519, "file_size": 38583104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224546, "largest_seqno": 4178535373, "table_properties": {"data_size": 32728848, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909408, "raw_average_key_size": 28, "raw_value_size": 108308356, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925336, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357228, "oldest_key_time": 0, "file_creation_time": 1732357228, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274519, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:32.368334 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst) /dev/shm/ToplingTemp/Topling-147862-Em3kuN 2024/11/23-18:20:33.399670 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst): first pass time = 1.03's, 130.145'MB/sec 2024/11/23-18:20:35.495349 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst): old prealloc_size = 147639500, real_size = 34071168 2024/11/23-18:20:35.495932 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927989 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.63 avg-zval = 34.55 2024/11/23-18:20:35.496714 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:36.515123 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274520.sst): second pass time = 2.09's, 51.688'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 37.320'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3148.151'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.23's, 162.461'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31520.528'MB/sec zip my value time = 2.09's, unzip length = 0.108'GB zip my value throughput = 51.688'MB/sec zip pipeline throughput = 146.659'MB/sec entries = 927989 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.63 avg-zval = 34.55 usrkeys = 927989 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.63 avg-zval = 34.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927989 bytes = 3131984 } seq expand size = 6495923 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4432 value = 3.3758 dict = 1.00 all = 3.5268 } Zip/UnZip{ index = 0.0958 value = 0.2962 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 9.078135 GB avg = 0.167 KB (by entry num) total key len = 1.492714 GB avg = 0.028 KB total ukey len = 1.034945 GB avg = 0.015 KB total ukey num = 0.067403760 Billion total entry num = 0.054211489 Billion write speed all = 0.311601109 MB/sec (with seq num) write speed all = 0.298817001 MB/sec (without seq num) 2024/11/23-18:20:36.567706 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274520: 927989 keys, 38056364 bytes, temperature: kWarm 2024/11/23-18:20:36.567797 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357236567749, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274520, "file_size": 38056364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024708, "largest_seqno": 4178535404, "table_properties": {"data_size": 32061744, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983692, "raw_average_key_size": 28, "raw_value_size": 108234038, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927989, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357232, "oldest_key_time": 0, "file_creation_time": 1732357232, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274520, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:36.632014 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst) /dev/shm/ToplingTemp/Topling-147862-TG7uHP 2024/11/23-18:20:37.673028 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst): first pass time = 1.04's, 128.935'MB/sec 2024/11/23-18:20:39.486607 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst): old prealloc_size = 147639500, real_size = 34355344 2024/11/23-18:20:39.487150 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926886 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.81 avg-zval = 34.91 2024/11/23-18:20:39.487898 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:40.542239 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274521.sst): second pass time = 1.81's, 59.736'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 36.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.118'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.16's, 246.715'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34067.056'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.736'MB/sec zip pipeline throughput = 111.204'MB/sec entries = 926886 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 34.91 usrkeys = 926886 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 34.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926886 bytes = 3128256 } seq expand size = 6488202 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5160 value = 3.3456 dict = 1.00 all = 3.5009 } Zip/UnZip{ index = 0.0951 value = 0.2989 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 9.192889 GB avg = 0.167 KB (by entry num) total key len = 1.518667 GB avg = 0.028 KB total ukey len = 1.053482 GB avg = 0.015 KB total ukey num = 0.068330646 Billion total entry num = 0.055138375 Billion write speed all = 0.315711279 MB/sec (with seq num) write speed all = 0.302710137 MB/sec (without seq num) 2024/11/23-18:20:40.610822 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274521: 926886 keys, 38337840 bytes, temperature: kWarm 2024/11/23-18:20:40.610931 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357240610841, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274521, "file_size": 38337840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4178535448, "table_properties": {"data_size": 32360640, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952808, "raw_average_key_size": 28, "raw_value_size": 108265040, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926886, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357236, "oldest_key_time": 0, "file_creation_time": 1732357236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274521, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:40.651257 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst) /dev/shm/ToplingTemp/Topling-147862-KxUoHR 2024/11/23-18:20:41.575929 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst): first pass time = 0.92's, 145.158'MB/sec 2024/11/23-18:20:43.166207 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst): old prealloc_size = 147639500, real_size = 35990960 2024/11/23-18:20:43.166760 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929736 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.36 avg-zval = 36.55 2024/11/23-18:20:43.168491 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:44.164856 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274522.sst): second pass time = 1.59's, 68.063'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 40.389'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.853'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.13's, 309.404'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34673.822'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.063'MB/sec zip pipeline throughput = 68.063'MB/sec entries = 929736 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.36 avg-zval = 36.55 usrkeys = 929736 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.36 avg-zval = 36.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929736 bytes = 3486528 } seq expand size = 6508152 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.4440 value = 3.1840 dict = 1.00 all = 3.3277 } Zip/UnZip{ index = 0.0957 value = 0.3141 dict = 1.00 all = 0.3005 } ---------------------------- total value len = 9.307582 GB avg = 0.166 KB (by entry num) total key len = 1.544700 GB avg = 0.028 KB total ukey len = 1.072077 GB avg = 0.015 KB total ukey num = 0.069260382 Billion total entry num = 0.056068111 Billion write speed all = 0.319824874 MB/sec (with seq num) write speed all = 0.306605920 MB/sec (without seq num) 2024/11/23-18:20:44.213812 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274522: 929736 keys, 40333524 bytes, temperature: kWarm 2024/11/23-18:20:44.213883 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357244213847, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274522, "file_size": 40333524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4178535475, "table_properties": {"data_size": 33977952, "index_size": 1780416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032608, "raw_average_key_size": 28, "raw_value_size": 108185264, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929736, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357240, "oldest_key_time": 0, "file_creation_time": 1732357240, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274522, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:44.250501 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst) /dev/shm/ToplingTemp/Topling-147862-POtvqO 2024/11/23-18:20:45.327032 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst): first pass time = 1.08's, 124.681'MB/sec 2024/11/23-18:20:46.895374 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst): old prealloc_size = 147639500, real_size = 36447136 2024/11/23-18:20:46.896081 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934311 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0344 GB avg-val = 115.65 avg-zval = 36.83 2024/11/23-18:20:46.896797 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:47.804415 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274523.sst): second pass time = 1.57's, 68.935'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 43.785'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2623.057'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.16's, 251.647'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35736.042'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.935'MB/sec zip pipeline throughput = 68.935'MB/sec entries = 934311 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.65 avg-zval = 36.83 usrkeys = 934311 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.36 avg-zval = 36.83 TagRS{ kinds = RS_Key0_TagN bytes = 146080 } TagArray{ size = 627519 bytes = 2353216 } seq expand size = 4392633 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3768 value = 3.1400 dict = 1.00 all = 3.3722 } Zip/UnZip{ index = 0.0964 value = 0.3185 dict = 1.00 all = 0.2965 } ---------------------------- total value len = 9.420032 GB avg = 0.165 KB (by entry num) total key len = 1.570860 GB avg = 0.028 KB total ukey len = 1.090763 GB avg = 0.016 KB total ukey num = 0.070194693 Billion total entry num = 0.057002422 Billion write speed all = 0.323875091 MB/sec (with seq num) write speed all = 0.310437299 MB/sec (without seq num) 2024/11/23-18:20:47.855016 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274523: 934311 keys, 39801054 bytes, temperature: kWarm 2024/11/23-18:20:47.855091 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357247855055, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274523, "file_size": 39801054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34412624, "index_size": 1800768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26160708, "raw_average_key_size": 28, "raw_value_size": 108057154, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934311, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357244, "oldest_key_time": 0, "file_creation_time": 1732357244, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274523, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:47.891690 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst) /dev/shm/ToplingTemp/Topling-147862-Fe8jfQ 2024/11/23-18:20:49.111534 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst): first pass time = 1.22's, 110.032'MB/sec 2024/11/23-18:20:50.616311 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst): old prealloc_size = 147639500, real_size = 33075120 2024/11/23-18:20:50.617023 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926116 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.93 avg-zval = 33.55 2024/11/23-18:20:50.617889 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:51.583731 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274524.sst): second pass time = 1.49's, 72.641'MB/sec, value only(80.7% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.97's, 38.543'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2590.615'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.20's, 190.690'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35620.187'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.641'MB/sec zip pipeline throughput = 72.641'MB/sec entries = 926116 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.93 avg-zval = 33.55 usrkeys = 926116 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.93 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926116 bytes = 3357184 } seq expand size = 6482812 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4466 value = 3.4852 dict = 1.00 all = 3.5996 } Zip/UnZip{ index = 0.0957 value = 0.2869 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 9.534801 GB avg = 0.165 KB (by entry num) total key len = 1.596792 GB avg = 0.028 KB total ukey len = 1.109285 GB avg = 0.016 KB total ukey num = 0.071120809 Billion total entry num = 0.057928538 Billion write speed all = 0.327984671 MB/sec (with seq num) write speed all = 0.314330077 MB/sec (without seq num) 2024/11/23-18:20:51.629857 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274524: 926116 keys, 37287246 bytes, temperature: kWarm 2024/11/23-18:20:51.629929 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357251629890, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274524, "file_size": 37287246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3624371836, "largest_seqno": 4012775935, "table_properties": {"data_size": 31070368, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931248, "raw_average_key_size": 28, "raw_value_size": 108286524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926116, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357247, "oldest_key_time": 0, "file_creation_time": 1732357247, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274524, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:51.665753 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst) /dev/shm/ToplingTemp/Topling-147862-oD2E3N 2024/11/23-18:20:52.799864 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst): first pass time = 1.13's, 118.350'MB/sec 2024/11/23-18:20:54.440786 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst): old prealloc_size = 147639500, real_size = 36033744 2024/11/23-18:20:54.441391 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925301 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.05 avg-zval = 36.78 2024/11/23-18:20:54.442174 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:55.291529 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274525.sst): second pass time = 1.64's, 66.035'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3034.436'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 315.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35227.880'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.035'MB/sec zip pipeline throughput = 66.035'MB/sec entries = 925301 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.05 avg-zval = 36.78 usrkeys = 925301 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.05 avg-zval = 36.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925301 bytes = 3354224 } seq expand size = 6477107 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4438 value = 3.1827 dict = 1.00 all = 3.3352 } Zip/UnZip{ index = 0.0958 value = 0.3142 dict = 1.00 all = 0.2998 } ---------------------------- total value len = 9.649588 GB avg = 0.164 KB (by entry num) total key len = 1.622700 GB avg = 0.028 KB total ukey len = 1.127792 GB avg = 0.016 KB total ukey num = 0.072046110 Billion total entry num = 0.058853839 Billion write speed all = 0.332093869 MB/sec (with seq num) write speed all = 0.318222683 MB/sec (without seq num) 2024/11/23-18:20:55.341354 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274525: 925301 keys, 40243092 bytes, temperature: kWarm 2024/11/23-18:20:55.341428 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357255341390, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274525, "file_size": 40243092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 34030272, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908428, "raw_average_key_size": 28, "raw_value_size": 108309394, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925301, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357251, "oldest_key_time": 0, "file_creation_time": 1732357251, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274525, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:55.377110 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst) /dev/shm/ToplingTemp/Topling-147862-B7Z10N 2024/11/23-18:20:56.450376 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst): first pass time = 1.07's, 125.060'MB/sec 2024/11/23-18:20:57.987058 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst): old prealloc_size = 147639500, real_size = 33004480 2024/11/23-18:20:57.987616 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927999 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.63 avg-zval = 33.41 2024/11/23-18:20:57.989518 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:20:58.851868 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274526.sst): second pass time = 1.54's, 70.496'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3296.553'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 287.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33383.003'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.496'MB/sec zip pipeline throughput = 70.496'MB/sec entries = 927999 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 33.41 usrkeys = 927999 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927999 bytes = 3480016 } seq expand size = 6495993 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5141 value = 3.4906 dict = 1.00 all = 3.5947 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 9.764318 GB avg = 0.163 KB (by entry num) total key len = 1.648684 GB avg = 0.028 KB total ukey len = 1.146351 GB avg = 0.016 KB total ukey num = 0.072974109 Billion total entry num = 0.059781838 Billion write speed all = 0.336204186 MB/sec (with seq num) write speed all = 0.322115760 MB/sec (without seq num) 2024/11/23-18:20:58.897731 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274526: 927999 keys, 37338160 bytes, temperature: kWarm 2024/11/23-18:20:58.897800 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357258897767, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274526, "file_size": 37338160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 31007024, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983972, "raw_average_key_size": 28, "raw_value_size": 108233858, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927999, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357255, "oldest_key_time": 0, "file_creation_time": 1732357255, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274526, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:20:58.936308 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst) /dev/shm/ToplingTemp/Topling-147862-TKkurP 2024/11/23-18:21:00.025463 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst): first pass time = 1.09's, 123.237'MB/sec 2024/11/23-18:21:01.692022 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst): old prealloc_size = 147639500, real_size = 33094640 2024/11/23-18:21:01.692617 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930517 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.24 avg-zval = 33.39 2024/11/23-18:21:01.694386 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:02.522301 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274527.sst): second pass time = 1.67's, 64.934'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.442'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3131.435'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 277.001'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33465.520'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.934'MB/sec zip pipeline throughput = 64.934'MB/sec entries = 930517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.24 avg-zval = 33.39 usrkeys = 930517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.86 avg-zval = 33.39 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747520 bytes = 2803216 } seq expand size = 5232640 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3782 value = 3.4814 dict = 1.00 all = 3.6375 } Zip/UnZip{ index = 0.0964 value = 0.2872 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 9.877714 GB avg = 0.163 KB (by entry num) total key len = 1.674738 GB avg = 0.028 KB total ukey len = 1.164962 GB avg = 0.016 KB total ukey num = 0.073904626 Billion total entry num = 0.060712355 Billion write speed all = 0.340275325 MB/sec (with seq num) write speed all = 0.325969156 MB/sec (without seq num) 2024/11/23-18:21:02.568591 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274527: 930517 keys, 36898630 bytes, temperature: kWarm 2024/11/23-18:21:02.568668 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357262568633, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274527, "file_size": 36898630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 31068640, "index_size": 1793216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054476, "raw_average_key_size": 28, "raw_value_size": 108163302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357258, "oldest_key_time": 0, "file_creation_time": 1732357258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274527, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:02.604684 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst) /dev/shm/ToplingTemp/Topling-147862-weHWWP 2024/11/23-18:21:03.718958 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst): first pass time = 1.11's, 120.459'MB/sec 2024/11/23-18:21:05.231477 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst): old prealloc_size = 147639500, real_size = 30785152 2024/11/23-18:21:05.232095 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932826 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 115.88 avg-zval = 30.85 2024/11/23-18:21:05.233638 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:06.068024 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274528.sst): second pass time = 1.51's, 71.552'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.861'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2976.320'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 272.139'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 41627.636'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.552'MB/sec zip pipeline throughput = 71.552'MB/sec entries = 932826 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.88 avg-zval = 30.85 usrkeys = 932826 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.88 avg-zval = 30.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932826 bytes = 3381504 } seq expand size = 6529782 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5131 value = 3.7564 dict = 1.00 all = 3.8329 } Zip/UnZip{ index = 0.0951 value = 0.2662 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 9.992342 GB avg = 0.162 KB (by entry num) total key len = 1.700858 GB avg = 0.028 KB total ukey len = 1.183618 GB avg = 0.016 KB total ukey num = 0.074837452 Billion total entry num = 0.061645181 Billion write speed all = 0.344385054 MB/sec (with seq num) write speed all = 0.329860592 MB/sec (without seq num) 2024/11/23-18:21:06.110887 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274528: 932826 keys, 35017250 bytes, temperature: kWarm 2024/11/23-18:21:06.110957 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357266110921, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274528, "file_size": 35017250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 28777200, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26119128, "raw_average_key_size": 28, "raw_value_size": 108098732, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357262, "oldest_key_time": 0, "file_creation_time": 1732357262, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274528, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:06.155506 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst) /dev/shm/ToplingTemp/Topling-147862-nznKAR 2024/11/23-18:21:07.024640 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst): first pass time = 0.87's, 154.435'MB/sec 2024/11/23-18:21:08.338899 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst): old prealloc_size = 147639500, real_size = 32680368 2024/11/23-18:21:08.339517 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931664 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.06 avg-zval = 32.90 2024/11/23-18:21:08.340355 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:09.199035 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274529.sst): second pass time = 1.31's, 82.390'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.912'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3052.422'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 280.912'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34369.960'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.390'MB/sec zip pipeline throughput = 82.390'MB/sec entries = 931664 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.06 avg-zval = 32.90 usrkeys = 931664 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.06 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931664 bytes = 3377296 } seq expand size = 6521648 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3773 value = 3.5277 dict = 1.00 all = 3.6365 } Zip/UnZip{ index = 0.0964 value = 0.2835 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 10.106995 GB avg = 0.162 KB (by entry num) total key len = 1.726944 GB avg = 0.028 KB total ukey len = 1.202252 GB avg = 0.016 KB total ukey num = 0.075769116 Billion total entry num = 0.062576845 Billion write speed all = 0.348497940 MB/sec (with seq num) write speed all = 0.333755325 MB/sec (without seq num) 2024/11/23-18:21:09.245303 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274529: 931664 keys, 36908764 bytes, temperature: kWarm 2024/11/23-18:21:09.245374 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357269245338, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274529, "file_size": 36908764, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516757455, "largest_seqno": 4012776268, "table_properties": {"data_size": 30651680, "index_size": 1795584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086592, "raw_average_key_size": 28, "raw_value_size": 108131256, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931664, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357266, "oldest_key_time": 0, "file_creation_time": 1732357266, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274529, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:09.283968 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst) /dev/shm/ToplingTemp/Topling-147862-kjpzMN 2024/11/23-18:21:10.313995 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst): first pass time = 1.03's, 130.315'MB/sec 2024/11/23-18:21:11.642774 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst): old prealloc_size = 147639500, real_size = 34026128 2024/11/23-18:21:11.643334 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922525 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.49 avg-zval = 34.72 2024/11/23-18:21:11.644182 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:12.489284 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274530.sst): second pass time = 1.33's, 81.639'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.298'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.146'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 297.857'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32435.702'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.639'MB/sec zip pipeline throughput = 81.639'MB/sec entries = 922525 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.49 avg-zval = 34.72 usrkeys = 922525 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.49 avg-zval = 34.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922525 bytes = 3459488 } seq expand size = 6457675 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4426 value = 3.3841 dict = 1.00 all = 3.5003 } Zip/UnZip{ index = 0.0958 value = 0.2955 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 10.221840 GB avg = 0.161 KB (by entry num) total key len = 1.752775 GB avg = 0.028 KB total ukey len = 1.220702 GB avg = 0.016 KB total ukey num = 0.076691641 Billion total entry num = 0.063499370 Billion write speed all = 0.352606529 MB/sec (with seq num) write speed all = 0.337648023 MB/sec (without seq num) 2024/11/23-18:21:12.535422 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274530: 922525 keys, 38344136 bytes, temperature: kWarm 2024/11/23-18:21:12.535493 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357272535457, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274530, "file_size": 38344136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 32028480, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830700, "raw_average_key_size": 28, "raw_value_size": 108387078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357269, "oldest_key_time": 0, "file_creation_time": 1732357269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274530, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:12.573851 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst) /dev/shm/ToplingTemp/Topling-147862-uFf1qR 2024/11/23-18:21:13.575914 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst): first pass time = 1.00's, 133.947'MB/sec 2024/11/23-18:21:14.922523 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst): old prealloc_size = 147639500, real_size = 33600848 2024/11/23-18:21:14.923084 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933045 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.85 avg-zval = 33.85 2024/11/23-18:21:14.923880 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:15.773841 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274531.sst): second pass time = 1.35's, 80.319'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.442'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.020'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 296.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33309.658'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.319'MB/sec zip pipeline throughput = 80.319'MB/sec entries = 933045 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.85 avg-zval = 33.85 usrkeys = 933045 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.85 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933045 bytes = 3382304 } seq expand size = 6531315 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4455 value = 3.4227 dict = 1.00 all = 3.5476 } Zip/UnZip{ index = 0.0957 value = 0.2922 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 10.336464 GB avg = 0.160 KB (by entry num) total key len = 1.778900 GB avg = 0.028 KB total ukey len = 1.239363 GB avg = 0.016 KB total ukey num = 0.077624686 Billion total entry num = 0.064432415 Billion write speed all = 0.356716551 MB/sec (with seq num) write speed all = 0.341539717 MB/sec (without seq num) 2024/11/23-18:21:15.820610 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274531: 933045 keys, 37833526 bytes, temperature: kWarm 2024/11/23-18:21:15.820680 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357275820644, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274531, "file_size": 37833526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 31580928, "index_size": 1786496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26125260, "raw_average_key_size": 28, "raw_value_size": 108092574, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933045, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357272, "oldest_key_time": 0, "file_creation_time": 1732357272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274531, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:15.860960 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst) /dev/shm/ToplingTemp/Topling-147862-anC4TN 2024/11/23-18:21:16.867680 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst): first pass time = 1.01's, 133.329'MB/sec 2024/11/23-18:21:18.170174 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst): old prealloc_size = 147639500, real_size = 32763440 2024/11/23-18:21:18.170743 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926009 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.94 avg-zval = 33.23 2024/11/23-18:21:18.171590 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:19.008508 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274532.sst): second pass time = 1.30's, 83.194'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.245'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.161'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 291.486'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35418.065'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.194'MB/sec zip pipeline throughput = 83.194'MB/sec entries = 926009 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.23 usrkeys = 926009 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926009 bytes = 3472544 } seq expand size = 6482063 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5125 value = 3.5193 dict = 1.00 all = 3.6186 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 10.451235 GB avg = 0.160 KB (by entry num) total key len = 1.804828 GB avg = 0.028 KB total ukey len = 1.257883 GB avg = 0.016 KB total ukey num = 0.078550695 Billion total entry num = 0.065358424 Billion write speed all = 0.360824859 MB/sec (with seq num) write speed all = 0.345431372 MB/sec (without seq num) 2024/11/23-18:21:19.053467 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274532: 926009 keys, 37090996 bytes, temperature: kWarm 2024/11/23-18:21:19.053536 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357279053500, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274532, "file_size": 37090996, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 30770016, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928252, "raw_average_key_size": 28, "raw_value_size": 108289574, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926009, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357275, "oldest_key_time": 0, "file_creation_time": 1732357275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274532, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:19.090557 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst) /dev/shm/ToplingTemp/Topling-147862-f8WsTP 2024/11/23-18:21:19.973608 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst): first pass time = 0.88's, 151.999'MB/sec 2024/11/23-18:21:21.294767 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst): old prealloc_size = 147639500, real_size = 32942224 2024/11/23-18:21:21.295342 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925220 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.07 avg-zval = 33.43 2024/11/23-18:21:21.296170 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:22.136230 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274533.sst): second pass time = 1.32's, 82.067'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.007'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 296.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33516.193'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.067'MB/sec zip pipeline throughput = 82.067'MB/sec entries = 925220 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.07 avg-zval = 33.43 usrkeys = 925220 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.07 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925220 bytes = 3469584 } seq expand size = 6476540 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3750 value = 3.5022 dict = 1.00 all = 3.6015 } Zip/UnZip{ index = 0.0964 value = 0.2855 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 10.566023 GB avg = 0.159 KB (by entry num) total key len = 1.830735 GB avg = 0.028 KB total ukey len = 1.276388 GB avg = 0.016 KB total ukey num = 0.079475915 Billion total entry num = 0.066283644 Billion write speed all = 0.364933371 MB/sec (with seq num) write speed all = 0.349323410 MB/sec (without seq num) 2024/11/23-18:21:22.180851 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274533: 925220 keys, 37266748 bytes, temperature: kWarm 2024/11/23-18:21:22.180924 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357282180884, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274533, "file_size": 37266748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30927168, "index_size": 1783552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906160, "raw_average_key_size": 28, "raw_value_size": 108311692, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925220, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357279, "oldest_key_time": 0, "file_creation_time": 1732357279, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274533, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:22.217569 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst) /dev/shm/ToplingTemp/Topling-147862-xdkoGO 2024/11/23-18:21:23.154164 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst): first pass time = 0.94's, 143.310'MB/sec 2024/11/23-18:21:24.608564 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst): old prealloc_size = 147639500, real_size = 35902752 2024/11/23-18:21:24.609090 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930405 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.26 avg-zval = 36.44 2024/11/23-18:21:24.609411 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:25.344201 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274534.sst): second pass time = 1.45's, 74.412'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 51.416'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3527.849'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.380'MB/sec write SST data time = 0.12's, 306.704'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35360.149'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.412'MB/sec zip pipeline throughput = 74.412'MB/sec entries = 930405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 36.44 usrkeys = 930405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.38 avg-zval = 36.44 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 282313 bytes = 917536 } seq expand size = 1976191 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5120 value = 3.1908 dict = 1.00 all = 3.5486 } Zip/UnZip{ index = 0.0951 value = 0.3134 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 10.676166 GB avg = 0.159 KB (by entry num) total key len = 1.856786 GB avg = 0.028 KB total ukey len = 1.294996 GB avg = 0.016 KB total ukey num = 0.080406320 Billion total entry num = 0.067214049 Billion write speed all = 0.368907787 MB/sec (with seq num) write speed all = 0.353080208 MB/sec (without seq num) 2024/11/23-18:21:25.389571 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274534: 930405 keys, 37822257 bytes, temperature: kWarm 2024/11/23-18:21:25.389641 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357285389604, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274534, "file_size": 37822257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33899792, "index_size": 1770176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051340, "raw_average_key_size": 28, "raw_value_size": 108166522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357282, "oldest_key_time": 0, "file_creation_time": 1732357282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274534, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:25.427719 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst) /dev/shm/ToplingTemp/Topling-147862-soutbP 2024/11/23-18:21:26.326302 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst): first pass time = 0.90's, 149.372'MB/sec 2024/11/23-18:21:27.815585 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst): old prealloc_size = 147639500, real_size = 37311344 2024/11/23-18:21:27.816137 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925707 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 116.99 avg-zval = 38.14 2024/11/23-18:21:27.816533 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:28.537924 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274535.sst): second pass time = 1.49's, 72.755'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 54.458'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3335.865'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.387'MB/sec write SST data time = 0.12's, 318.981'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33624.759'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.755'MB/sec zip pipeline throughput = 72.755'MB/sec entries = 925707 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 38.14 usrkeys = 925707 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.05 avg-zval = 38.14 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 272206 bytes = 1020784 } seq expand size = 1905442 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4461 value = 3.0673 dict = 1.00 all = 3.4121 } Zip/UnZip{ index = 0.0957 value = 0.3260 dict = 1.00 all = 0.2931 } ---------------------------- total value len = 10.786370 GB avg = 0.158 KB (by entry num) total key len = 1.882706 GB avg = 0.028 KB total ukey len = 1.313510 GB avg = 0.016 KB total ukey num = 0.081332027 Billion total entry num = 0.068139756 Billion write speed all = 0.372879523 MB/sec (with seq num) write speed all = 0.356835467 MB/sec (without seq num) 2024/11/23-18:21:28.586530 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274535: 925707 keys, 39335425 bytes, temperature: kWarm 2024/11/23-18:21:28.586604 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357288586566, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274535, "file_size": 39335425, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4151861871, "table_properties": {"data_size": 35307360, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919796, "raw_average_key_size": 28, "raw_value_size": 108297942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357285, "oldest_key_time": 0, "file_creation_time": 1732357285, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274535, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:28.636399 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst) /dev/shm/ToplingTemp/Topling-147862-Tcgu9P 2024/11/23-18:21:29.594521 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst): first pass time = 0.96's, 140.091'MB/sec 2024/11/23-18:21:30.931311 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst): old prealloc_size = 147639500, real_size = 34754896 2024/11/23-18:21:30.931829 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928496 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.55 avg-zval = 35.28 2024/11/23-18:21:30.932593 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:31.768040 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274536.sst): second pass time = 1.34's, 81.000'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3550.046'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 311.936'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32063.801'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.000'MB/sec zip pipeline throughput = 81.000'MB/sec entries = 928496 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 35.28 usrkeys = 928496 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 35.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928496 bytes = 3133696 } seq expand size = 6499472 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5140 value = 3.3038 dict = 1.00 all = 3.4644 } Zip/UnZip{ index = 0.0951 value = 0.3027 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 10.901089 GB avg = 0.158 KB (by entry num) total key len = 1.908704 GB avg = 0.028 KB total ukey len = 1.332080 GB avg = 0.016 KB total ukey num = 0.082260523 Billion total entry num = 0.069068252 Billion write speed all = 0.376985310 MB/sec (with seq num) write speed all = 0.360724178 MB/sec (without seq num) 2024/11/23-18:21:31.818201 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274536: 928496 keys, 38741780 bytes, temperature: kWarm 2024/11/23-18:21:31.818276 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357291818240, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274536, "file_size": 38741780, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4178535298, "table_properties": {"data_size": 32756352, "index_size": 1766208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997888, "raw_average_key_size": 28, "raw_value_size": 108219892, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928496, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357288, "oldest_key_time": 0, "file_creation_time": 1732357288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274536, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:31.854368 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst) /dev/shm/ToplingTemp/Topling-147862-MNzdSN 2024/11/23-18:21:32.829547 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst): first pass time = 0.98's, 137.639'MB/sec 2024/11/23-18:21:34.142625 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst): old prealloc_size = 147639500, real_size = 34601008 2024/11/23-18:21:34.143171 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927217 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.75 avg-zval = 35.16 2024/11/23-18:21:34.143919 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:21:34.977547 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274537.sst): second pass time = 1.31's, 82.517'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.215'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.637'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 307.954'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31736.717'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.517'MB/sec zip pipeline throughput = 82.517'MB/sec entries = 927217 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 35.16 usrkeys = 927217 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 35.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927217 bytes = 3129376 } seq expand size = 6490519 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5121 value = 3.3202 dict = 1.00 all = 3.4785 } Zip/UnZip{ index = 0.0951 value = 0.3012 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 11.015835 GB avg = 0.157 KB (by entry num) total key len = 1.934666 GB avg = 0.028 KB total ukey len = 1.350624 GB avg = 0.016 KB total ukey num = 0.083187740 Billion total entry num = 0.069995469 Billion write speed all = 0.381090283 MB/sec (with seq num) write speed all = 0.364612407 MB/sec (without seq num) 2024/11/23-18:21:35.028132 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274537: 927217 keys, 38584940 bytes, temperature: kWarm 2024/11/23-18:21:35.028204 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357295028167, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274537, "file_size": 38584940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4104791485, "largest_seqno": 4178535139, "table_properties": {"data_size": 32604896, "index_size": 1764096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962076, "raw_average_key_size": 28, "raw_value_size": 108255658, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927217, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357291, "oldest_key_time": 0, "file_creation_time": 1732357291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274537, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:35.068438 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst) /dev/shm/ToplingTemp/Topling-147862-O4o8xR 2024/11/23-18:21:35.144088 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst): first pass time = 0.08's, 236.633'MB/sec 2024/11/23-18:21:35.317324 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst): old prealloc_size = 147639500, real_size = 5337424 2024/11/23-18:21:35.317450 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122546 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0145 GB zip-val = 0.0051 GB avg-val = 118.00 avg-zval = 41.38 2024/11/23-18:21:35.317589 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/23-18:21:35.425358 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00/274538.sst): second pass time = 0.17's, 83.825'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 54.088'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2258.290'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.337'MB/sec write SST data time = 0.02's, 344.987'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.151'MB, throughput = 5331.713'MB/sec zip my value time = 0.17's, unzip length = 0.014'GB zip my value throughput = 83.825'MB/sec zip pipeline throughput = 83.825'MB/sec entries = 122546 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 41.38 usrkeys = 122546 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 41.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122546 bytes = 382976 } seq expand size = 857822 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0025 GB value = 0.0145 GB dict = 0.15 MB all = 0.0179 GB } __ZipSize{ index = 0.0002 GB value = 0.0051 GB dict = 0.15 MB all = 0.0058 GB } UnZip/Zip{ index = 10.3923 value = 2.8517 dict = 1.00 all = 3.0623 } Zip/UnZip{ index = 0.0962 value = 0.3507 dict = 1.00 all = 0.3266 } ---------------------------- total value len = 11.031153 GB avg = 0.157 KB (by entry num) total key len = 1.938097 GB avg = 0.028 KB total ukey len = 1.353075 GB avg = 0.016 KB total ukey num = 0.083310286 Billion total entry num = 0.070118015 Billion write speed all = 0.381636990 MB/sec (with seq num) write speed all = 0.365130483 MB/sec (without seq num) 2024/11/23-18:21:35.438812 139957592835648 [compaction_job.cc:1948] [default] [JOB 32969] Generated table #274538: 122546 keys, 5842625 bytes, temperature: kWarm 2024/11/23-18:21:35.438865 139957592835648 EVENT_LOG_v1 {"time_micros": 1732357295438837, "cf_name": "default", "job": 32969, "event": "table_file_creation", "file_number": 274538, "file_size": 5842625, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4150783964, "largest_seqno": 4172876505, "table_properties": {"data_size": 5070784, "index_size": 235840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3431288, "raw_average_key_size": 28, "raw_value_size": 14460428, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 122546, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357295, "oldest_key_time": 0, "file_creation_time": 1732357295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274538, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:21:35.448659 139957592835648 [compaction_job.cc:703] job-32969: subcompact[0], size: 0.729667 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32969/att-00 [274519,274520,274521,274522,274523,274524,274525,274526,274527,274528,274529,274530,274531,274532,274533,274534,274535,274536,274537,274538] 2024/11/23-18:21:35.564586 139957592835648 (Original Log Time 2024/11/23-18:20:27.688292) EVENT_LOG_v1 {"time_micros": 1732357227688263, "job": 32969, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [274486], "files_L3": [274402, 274403, 274404, 274443, 274444, 274445, 274446, 274447, 274448, 274449, 274450, 274451, 274452, 274453, 274454, 274455, 274456, 274457, 274511], "score": 0.986053, "input_data_size": 753262466, "oldest_snapshot_seqno": 4178803291} 2024/11/23-18:21:35.564589 139957592835648 (Original Log Time 2024/11/23-18:21:35.563051) [compaction_job.cc:2022] [default] [JOB 32969] Compacted 1@2 + 19@3 files to L3 => 729667461 bytes 2024/11/23-18:21:35.564590 139957592835648 (Original Log Time 2024/11/23-18:21:35.564517) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.8 wr, level 3, files in(1, 19) out(20 +0 blob) MB in(27.1, 691.3 +0.0 blob) out(695.9 +0.0 blob), read-write-amplify(52.2) write-amplify(25.7) OK, records in: 17869865, records dropped: 110014 output_compression: Snappy 2024/11/23-18:21:35.564594 139957592835648 (Original Log Time 2024/11/23-18:21:35.564561) EVENT_LOG_v1 {"time_micros": 1732357295564540, "cf": "default", "job": 32969, "event": "compaction_finished", "compaction_time_micros": 67760373, "compaction_time_cpu_micros": 28213130, "output_level": 3, "num_output_files": 20, "total_output_size": 729667461, "num_input_records": 17869865, "num_output_records": 17759851, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 539167196, "file_range_sync_nanos": 0, "file_fsync_nanos": 13810, "file_prepare_write_nanos": 54172, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:21:35.713156 139957592835648 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732357227,"executesMs":67999,"compactionJobId":32969,"attempt":0,"compactionInputRawBytes":2583652094,"compactionInputZipBytes":753262466,"compactionOutputRawBytes":0,"compactionOutputZipBytes":729667461,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-dce67559f786bf4327815e0fe615dacf-1c2eee8b631f35e7-00"}}, response = 2024/11/23-18:21:35.713250 139957592835648 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32969/att-00: olev 3, work 68.000 s, result 7.646 ms, install 1.557 ms, input{raw 2.406 GiB zip 718.367 MiB}