2024/11/24-04:20:49.151816 139957466945088 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00: fork to child time = 0.007873 sec 2024/11/24-04:20:49.152585 139957466945088 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00 2024/11/24-04:20:49.158543 139957466945088 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00 2024/11/24-04:20:49.159470 139957466945088 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:20:49.159749 139957466945088 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:20:49.212885 139957466945088 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41422/att-00/MANIFEST-206347 2024/11/24-04:20:49.229332 139957466945088 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41422/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 291740, last_sequence is 4436649780, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 291589 2024/11/24-04:20:49.229340 139957466945088 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:20:49.229455 139957466945088 [version_set.cc:5721] Creating manifest 291740 2024/11/24-04:20:49.263158 139957466945088 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:20:49.284534 139957466945088 [compaction_job.cc:2437] [default] [JOB 41422] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/24-04:20:49.284552 139957466945088 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31034 Base level 2, inputs: [291737(27MB)], [291680(35MB) 291681(34MB) 291682(33MB) 291683(35MB) 291684(30MB) 291685(34MB) 291686(33MB) 291687(32MB) 291688(33MB) 291689(32MB) 291690(33MB) 291691(32MB) 291692(33MB) 291693(32MB) 291694(33MB) 291695(32MB) 291696(33MB) 291697(32MB) 291698(34MB) 291699(32MB)] 2024/11/24-04:20:49.291942 139957466945088 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:20:49.291970 139957466945088 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:20:49.292004 139957466945088 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:20:49.292010 139957466945088 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:20:49.292021 139957466945088 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:20:49.870225 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst) /dev/shm/ToplingTemp/Topling-267937-KCu9Jj 2024/11/24-04:20:50.448396 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst): first pass time = 0.58's, 232.161'MB/sec 2024/11/24-04:20:52.084138 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst): old prealloc_size = 147639500, real_size = 33446848 2024/11/24-04:20:52.084754 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927196 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.76 avg-zval = 33.92 2024/11/24-04:20:52.085637 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:20:53.530083 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291741.sst): second pass time = 1.63's, 66.519'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.45's, 25.967'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3013.994'MB/sec (index lex order gen) rebuild zvType time = 1.18's, 0.196'MB/sec write SST data time = 0.27's, 141.605'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36984.527'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.519'MB/sec zip pipeline throughput = 66.757'MB/sec entries = 927196 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.92 usrkeys = 927196 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927196 bytes = 3245200 } seq expand size = 6490372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5130 value = 3.4421 dict = 1.00 all = 3.5747 } Zip/UnZip{ index = 0.0951 value = 0.2905 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 2251.440320 GB avg = 0.200 KB (by entry num) total key len = 306.908924 GB avg = 0.027 KB total ukey len = 237.373271 GB avg = 0.015 KB total ukey num = 15.739588359 Billion total entry num = 11.280902307 Billion write speed all = 36.578511986 MB/sec (with seq num) write speed all = 35.288184261 MB/sec (without seq num) 2024/11/24-04:20:53.585117 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291741: 927196 keys, 37546520 bytes, temperature: kWarm 2024/11/24-04:20:53.585255 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393253585165, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291741, "file_size": 37546520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4436635447, "table_properties": {"data_size": 31450992, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961488, "raw_average_key_size": 28, "raw_value_size": 108256360, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393249, "oldest_key_time": 0, "file_creation_time": 1732393249, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:20:53.657999 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst) /dev/shm/ToplingTemp/Topling-267937-x0BAck 2024/11/24-04:20:54.623092 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst): first pass time = 0.97's, 139.082'MB/sec 2024/11/24-04:20:56.161458 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst): old prealloc_size = 147639500, real_size = 32403200 2024/11/24-04:20:56.162039 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926757 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.83 avg-zval = 32.81 2024/11/24-04:20:56.162808 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:20:57.293636 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291742.sst): second pass time = 1.54's, 70.413'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 32.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.564'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.23's, 156.067'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37235.420'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.413'MB/sec zip pipeline throughput = 78.779'MB/sec entries = 926757 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 32.81 usrkeys = 926757 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926757 bytes = 3243664 } seq expand size = 6487299 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5157 value = 3.5605 dict = 1.00 all = 3.6771 } Zip/UnZip{ index = 0.0951 value = 0.2809 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 2251.555076 GB avg = 0.200 KB (by entry num) total key len = 306.934873 GB avg = 0.027 KB total ukey len = 237.391806 GB avg = 0.015 KB total ukey num = 15.740515116 Billion total entry num = 11.281829064 Billion write speed all = 36.578555445 MB/sec (with seq num) write speed all = 35.288191150 MB/sec (without seq num) 2024/11/24-04:20:57.352224 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291742: 926757 keys, 36501282 bytes, temperature: kWarm 2024/11/24-04:20:57.352315 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393257352269, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291742, "file_size": 36501282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4436635457, "table_properties": {"data_size": 30408688, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949196, "raw_average_key_size": 28, "raw_value_size": 108268630, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393253, "oldest_key_time": 0, "file_creation_time": 1732393253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:20:57.430458 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst) /dev/shm/ToplingTemp/Topling-267937-PboF4l 2024/11/24-04:20:58.436045 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst): first pass time = 1.01's, 133.480'MB/sec 2024/11/24-04:21:00.447683 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst): old prealloc_size = 147639500, real_size = 31378560 2024/11/24-04:21:00.448277 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920046 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 117.88 avg-zval = 31.94 2024/11/24-04:21:00.449067 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:01.649748 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291743.sst): second pass time = 2.01's, 53.955'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.20's, 29.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3072.945'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.237'MB/sec write SST data time = 0.23's, 152.997'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 25208.964'MB/sec zip my value time = 2.01's, unzip length = 0.108'GB zip my value throughput = 53.955'MB/sec zip pipeline throughput = 122.091'MB/sec entries = 920046 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.88 avg-zval = 31.94 usrkeys = 920046 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.88 avg-zval = 31.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920046 bytes = 3220176 } seq expand size = 6440322 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4452 value = 3.6907 dict = 1.00 all = 3.7854 } Zip/UnZip{ index = 0.0957 value = 0.2710 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2251.669973 GB avg = 0.200 KB (by entry num) total key len = 306.960634 GB avg = 0.027 KB total ukey len = 237.410207 GB avg = 0.015 KB total ukey num = 15.741435162 Billion total entry num = 11.282749110 Billion write speed all = 36.578288376 MB/sec (with seq num) write speed all = 35.287899214 MB/sec (without seq num) 2024/11/24-04:21:01.696038 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291743: 920046 keys, 35457102 bytes, temperature: kWarm 2024/11/24-04:21:01.696128 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393261696076, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291743, "file_size": 35457102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4288976201, "largest_seqno": 4436635468, "table_properties": {"data_size": 29386672, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25761288, "raw_average_key_size": 28, "raw_value_size": 108456456, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920046, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393257, "oldest_key_time": 0, "file_creation_time": 1732393257, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:01.758045 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst) /dev/shm/ToplingTemp/Topling-267937-RxIn1j 2024/11/24-04:21:02.684247 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst): first pass time = 0.93's, 144.921'MB/sec 2024/11/24-04:21:04.543471 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst): old prealloc_size = 147639500, real_size = 33235232 2024/11/24-04:21:04.544005 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927229 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.75 avg-zval = 33.68 2024/11/24-04:21:04.544806 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:05.418816 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291744.sst): second pass time = 1.86's, 58.253'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.196'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 281.960'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35882.017'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.253'MB/sec zip pipeline throughput = 78.643'MB/sec entries = 927229 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 33.68 usrkeys = 927229 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927229 bytes = 3245312 } seq expand size = 6490603 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4455 value = 3.4666 dict = 1.00 all = 3.5946 } Zip/UnZip{ index = 0.0957 value = 0.2885 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 2251.784719 GB avg = 0.200 KB (by entry num) total key len = 306.986597 GB avg = 0.027 KB total ukey len = 237.428752 GB avg = 0.015 KB total ukey num = 15.742362391 Billion total entry num = 11.283676339 Billion write speed all = 36.578328986 MB/sec (with seq num) write speed all = 35.287903311 MB/sec (without seq num) 2024/11/24-04:21:05.466753 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291744: 927229 keys, 37338520 bytes, temperature: kWarm 2024/11/24-04:21:05.466821 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393265466787, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291744, "file_size": 37338520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4436634351, "table_properties": {"data_size": 31227856, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962412, "raw_average_key_size": 28, "raw_value_size": 108255374, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927229, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393261, "oldest_key_time": 0, "file_creation_time": 1732393261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:05.504594 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst) /dev/shm/ToplingTemp/Topling-267937-Yg6Qem 2024/11/24-04:21:06.317080 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst): first pass time = 0.81's, 165.201'MB/sec 2024/11/24-04:21:07.886159 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst): old prealloc_size = 147639500, real_size = 27800928 2024/11/24-04:21:07.886706 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.25 2024/11/24-04:21:07.887482 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:08.732323 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291745.sst): second pass time = 1.57's, 68.708'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.621'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.784'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.14's, 233.167'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 24996.484'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.708'MB/sec zip pipeline throughput = 125.697'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.25 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4439 value = 4.1838 dict = 1.00 all = 4.2164 } Zip/UnZip{ index = 0.0957 value = 0.2390 dict = 1.00 all = 0.2372 } ---------------------------- total value len = 2251.899087 GB avg = 0.200 KB (by entry num) total key len = 307.013062 GB avg = 0.027 KB total ukey len = 237.447656 GB avg = 0.015 KB total ukey num = 15.743307587 Billion total entry num = 11.284621535 Billion write speed all = 36.578609611 MB/sec (with seq num) write speed all = 35.288136968 MB/sec (without seq num) 2024/11/24-04:21:08.772843 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291745: 945196 keys, 31832112 bytes, temperature: kWarm 2024/11/24-04:21:08.772903 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393268772871, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291745, "file_size": 31832112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268753488, "largest_seqno": 4368729564, "table_properties": {"data_size": 25754384, "index_size": 1810048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393265, "oldest_key_time": 0, "file_creation_time": 1732393265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:08.810381 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst) /dev/shm/ToplingTemp/Topling-267937-Ws4WRk 2024/11/24-04:21:09.672511 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst): first pass time = 0.86's, 155.688'MB/sec 2024/11/24-04:21:11.258389 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst): old prealloc_size = 147639500, real_size = 31763936 2024/11/24-04:21:11.258945 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936374 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0297 GB avg-val = 115.34 avg-zval = 31.77 2024/11/24-04:21:11.259730 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:12.114882 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291746.sst): second pass time = 1.58's, 68.149'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.904'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3325.260'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.322'MB/sec write SST data time = 0.13's, 278.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 29224.068'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.149'MB/sec zip pipeline throughput = 126.073'MB/sec entries = 936374 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.34 avg-zval = 31.77 usrkeys = 936374 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.34 avg-zval = 31.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936374 bytes = 3277328 } seq expand size = 6554618 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5118 value = 3.6305 dict = 1.00 all = 3.7397 } Zip/UnZip{ index = 0.0951 value = 0.2754 dict = 1.00 all = 0.2674 } ---------------------------- total value len = 2252.013641 GB avg = 0.200 KB (by entry num) total key len = 307.039281 GB avg = 0.027 KB total ukey len = 237.466383 GB avg = 0.015 KB total ukey num = 15.744243961 Billion total entry num = 11.285557909 Billion write speed all = 36.578853221 MB/sec (with seq num) write speed all = 35.288335897 MB/sec (without seq num) 2024/11/24-04:21:12.159386 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291746: 936374 keys, 35889862 bytes, temperature: kWarm 2024/11/24-04:21:12.159451 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393272159416, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291746, "file_size": 35889862, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4436635217, "table_properties": {"data_size": 29748112, "index_size": 1781568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26218472, "raw_average_key_size": 28, "raw_value_size": 107999364, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936374, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393268, "oldest_key_time": 0, "file_creation_time": 1732393268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:12.196498 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst) /dev/shm/ToplingTemp/Topling-267937-oWzx2k 2024/11/24-04:21:13.079956 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst): first pass time = 0.88's, 151.929'MB/sec 2024/11/24-04:21:14.638893 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst): old prealloc_size = 147639500, real_size = 31116304 2024/11/24-04:21:14.639441 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923340 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.36 avg-zval = 31.53 2024/11/24-04:21:14.640184 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:15.453430 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291747.sst): second pass time = 1.56's, 69.550'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.079'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.968'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 273.550'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28260.257'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.550'MB/sec zip pipeline throughput = 118.201'MB/sec entries = 923340 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.36 avg-zval = 31.53 usrkeys = 923340 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.36 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923340 bytes = 3116288 } seq expand size = 6463380 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4439 value = 3.7217 dict = 1.00 all = 3.8251 } Zip/UnZip{ index = 0.0957 value = 0.2687 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 2252.128469 GB avg = 0.200 KB (by entry num) total key len = 307.065134 GB avg = 0.027 KB total ukey len = 237.484850 GB avg = 0.015 KB total ukey num = 15.745167301 Billion total entry num = 11.286481249 Billion write speed all = 36.579118517 MB/sec (with seq num) write speed all = 35.288557194 MB/sec (without seq num) 2024/11/24-04:21:15.525850 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291747: 923340 keys, 35088596 bytes, temperature: kWarm 2024/11/24-04:21:15.525897 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393275525870, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291747, "file_size": 35088596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4312250229, "largest_seqno": 4436634361, "table_properties": {"data_size": 29117120, "index_size": 1768192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25853520, "raw_average_key_size": 28, "raw_value_size": 108364344, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923340, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393272, "oldest_key_time": 0, "file_creation_time": 1732393272, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:15.564688 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst) /dev/shm/ToplingTemp/Topling-267937-o4jlQl 2024/11/24-04:21:16.415006 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst): first pass time = 0.85's, 157.850'MB/sec 2024/11/24-04:21:17.938884 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst): old prealloc_size = 147639500, real_size = 28318160 2024/11/24-04:21:17.939441 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944197 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0263 GB avg-val = 114.15 avg-zval = 27.84 2024/11/24-04:21:17.940233 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:18.784390 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291748.sst): second pass time = 1.52's, 70.764'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 38.401'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3362.607'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.330'MB/sec write SST data time = 0.13's, 250.995'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 30158.146'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 70.764'MB/sec zip pipeline throughput = 122.359'MB/sec entries = 944197 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.15 avg-zval = 27.84 usrkeys = 944197 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.15 avg-zval = 27.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944197 bytes = 3304704 } seq expand size = 6609379 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0263 GB dict = 1.07 MB all = 0.0325 GB } UnZip/Zip{ index = 10.5128 value = 4.1004 dict = 1.00 all = 4.1341 } Zip/UnZip{ index = 0.0951 value = 0.2439 dict = 1.00 all = 0.2419 } ---------------------------- total value len = 2252.242859 GB avg = 0.200 KB (by entry num) total key len = 307.091572 GB avg = 0.027 KB total ukey len = 237.503734 GB avg = 0.015 KB total ukey num = 15.746111498 Billion total entry num = 11.287425446 Billion write speed all = 36.579389855 MB/sec (with seq num) write speed all = 35.288782012 MB/sec (without seq num) 2024/11/24-04:21:18.827264 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291748: 944197 keys, 32466350 bytes, temperature: kWarm 2024/11/24-04:21:18.827332 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393278827297, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291748, "file_size": 32466350, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4436635489, "table_properties": {"data_size": 26285632, "index_size": 1796288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26437516, "raw_average_key_size": 28, "raw_value_size": 107780346, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944197, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393275, "oldest_key_time": 0, "file_creation_time": 1732393275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:18.866507 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst) /dev/shm/ToplingTemp/Topling-267937-TnGkAl 2024/11/24-04:21:19.733786 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst): first pass time = 0.87's, 154.767'MB/sec 2024/11/24-04:21:21.312744 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst): old prealloc_size = 147639500, real_size = 31478144 2024/11/24-04:21:21.313300 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-04:21:21.315163 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:22.136395 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291749.sst): second pass time = 1.58's, 68.797'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.029'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.455'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 274.085'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31251.188'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.797'MB/sec zip pipeline throughput = 124.657'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4462 value = 3.6787 dict = 1.00 all = 3.7872 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2640 } ---------------------------- total value len = 2252.357771 GB avg = 0.200 KB (by entry num) total key len = 307.117312 GB avg = 0.027 KB total ukey len = 237.522120 GB avg = 0.015 KB total ukey num = 15.747030798 Billion total entry num = 11.288344746 Billion write speed all = 36.579647651 MB/sec (with seq num) write speed all = 35.288996530 MB/sec (without seq num) 2024/11/24-04:21:22.195163 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291749: 919300 keys, 35439986 bytes, temperature: kWarm 2024/11/24-04:21:22.195243 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393282195199, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291749, "file_size": 35439986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269816258, "largest_seqno": 4385889357, "table_properties": {"data_size": 29488048, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393278, "oldest_key_time": 0, "file_creation_time": 1732393278, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:22.235895 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst) /dev/shm/ToplingTemp/Topling-267937-hpq4Ll 2024/11/24-04:21:23.082131 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst): first pass time = 0.85's, 158.612'MB/sec 2024/11/24-04:21:24.616567 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst): old prealloc_size = 147639500, real_size = 29743408 2024/11/24-04:21:24.617162 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934308 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0277 GB avg-val = 115.65 avg-zval = 29.67 2024/11/24-04:21:24.617912 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:25.446752 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291750.sst): second pass time = 1.53's, 70.455'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.509'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.227'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 260.217'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32059.986'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.455'MB/sec zip pipeline throughput = 111.613'MB/sec entries = 934308 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.65 avg-zval = 29.67 usrkeys = 934308 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.65 avg-zval = 29.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934308 bytes = 3036512 } seq expand size = 6540156 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0277 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.4413 value = 3.8982 dict = 1.00 all = 3.9911 } Zip/UnZip{ index = 0.0958 value = 0.2565 dict = 1.00 all = 0.2506 } ---------------------------- total value len = 2252.472369 GB avg = 0.200 KB (by entry num) total key len = 307.143473 GB avg = 0.027 KB total ukey len = 237.540806 GB avg = 0.015 KB total ukey num = 15.747965106 Billion total entry num = 11.289279054 Billion write speed all = 36.579928708 MB/sec (with seq num) write speed all = 35.289231827 MB/sec (without seq num) 2024/11/24-04:21:25.491125 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291750: 934308 keys, 33628882 bytes, temperature: kWarm 2024/11/24-04:21:25.491193 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393285491158, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291750, "file_size": 33628882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4372074265, "largest_seqno": 4436634378, "table_properties": {"data_size": 27720032, "index_size": 1789632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26160624, "raw_average_key_size": 28, "raw_value_size": 108057228, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934308, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393282, "oldest_key_time": 0, "file_creation_time": 1732393282, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:25.528972 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst) /dev/shm/ToplingTemp/Topling-267937-QS8dmm 2024/11/24-04:21:26.416558 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst): first pass time = 0.89's, 151.229'MB/sec 2024/11/24-04:21:27.905696 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst): old prealloc_size = 147639500, real_size = 31849744 2024/11/24-04:21:27.906248 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920610 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0299 GB avg-val = 117.79 avg-zval = 32.44 2024/11/24-04:21:27.908271 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:28.722580 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291751.sst): second pass time = 1.49's, 72.862'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.392'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 283.923'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36751.610'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.862'MB/sec zip pipeline throughput = 93.129'MB/sec entries = 920610 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.79 avg-zval = 32.44 usrkeys = 920610 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.79 avg-zval = 32.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920610 bytes = 3222144 } seq expand size = 6444270 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5131 value = 3.6307 dict = 1.00 all = 3.7353 } Zip/UnZip{ index = 0.0951 value = 0.2754 dict = 1.00 all = 0.2677 } ---------------------------- total value len = 2252.587254 GB avg = 0.200 KB (by entry num) total key len = 307.169250 GB avg = 0.027 KB total ukey len = 237.559218 GB avg = 0.015 KB total ukey num = 15.748885716 Billion total entry num = 11.290199664 Billion write speed all = 36.580226417 MB/sec (with seq num) write speed all = 35.289484709 MB/sec (without seq num) 2024/11/24-04:21:28.767461 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291751: 920610 keys, 35931814 bytes, temperature: kWarm 2024/11/24-04:21:28.767538 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393288767497, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291751, "file_size": 35931814, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4436634547, "table_properties": {"data_size": 29868032, "index_size": 1751360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25777080, "raw_average_key_size": 28, "raw_value_size": 108440780, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920610, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393285, "oldest_key_time": 0, "file_creation_time": 1732393285, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291751, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:28.803148 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst) /dev/shm/ToplingTemp/Topling-267937-p8xCgm 2024/11/24-04:21:29.693658 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst): first pass time = 0.89's, 150.727'MB/sec 2024/11/24-04:21:31.118679 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst): old prealloc_size = 147639500, real_size = 30250336 2024/11/24-04:21:31.119258 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929693 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.37 avg-zval = 30.36 2024/11/24-04:21:31.121235 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:31.942878 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291752.sst): second pass time = 1.42's, 75.961'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.235'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 267.735'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36743.762'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.961'MB/sec zip pipeline throughput = 75.961'MB/sec entries = 929693 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.37 avg-zval = 30.36 usrkeys = 929693 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.37 avg-zval = 30.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929693 bytes = 3137728 } seq expand size = 6507851 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0342 GB } UnZip/Zip{ index = 10.3786 value = 3.8328 dict = 1.00 all = 3.9200 } Zip/UnZip{ index = 0.0964 value = 0.2609 dict = 1.00 all = 0.2551 } ---------------------------- total value len = 2252.701948 GB avg = 0.200 KB (by entry num) total key len = 307.195281 GB avg = 0.027 KB total ukey len = 237.577812 GB avg = 0.015 KB total ukey num = 15.749815409 Billion total entry num = 11.291129357 Billion write speed all = 36.580554030 MB/sec (with seq num) write speed all = 35.289765438 MB/sec (without seq num) 2024/11/24-04:21:31.984766 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291752: 929693 keys, 34239108 bytes, temperature: kWarm 2024/11/24-04:21:31.984835 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393291984801, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291752, "file_size": 34239108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4368985812, "largest_seqno": 4436634393, "table_properties": {"data_size": 28226192, "index_size": 1791552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26031404, "raw_average_key_size": 28, "raw_value_size": 108186382, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929693, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393288, "oldest_key_time": 0, "file_creation_time": 1732393288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291752, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:32.025696 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst) /dev/shm/ToplingTemp/Topling-267937-nOH6Ml 2024/11/24-04:21:32.966698 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst): first pass time = 0.94's, 142.639'MB/sec 2024/11/24-04:21:34.432544 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst): old prealloc_size = 147639500, real_size = 32277872 2024/11/24-04:21:34.433116 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921735 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.61 avg-zval = 32.87 2024/11/24-04:21:34.435028 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:35.258404 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291753.sst): second pass time = 1.47's, 73.996'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.032'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.828'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 290.137'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33584.390'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.996'MB/sec zip pipeline throughput = 73.996'MB/sec entries = 921735 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 32.87 usrkeys = 921735 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.61 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921735 bytes = 3226080 } seq expand size = 6452145 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5136 value = 3.5786 dict = 1.00 all = 3.6911 } Zip/UnZip{ index = 0.0951 value = 0.2794 dict = 1.00 all = 0.2709 } ---------------------------- total value len = 2252.816809 GB avg = 0.200 KB (by entry num) total key len = 307.221090 GB avg = 0.027 KB total ukey len = 237.596247 GB avg = 0.015 KB total ukey num = 15.750737144 Billion total entry num = 11.292051092 Billion write speed all = 36.580831043 MB/sec (with seq num) write speed all = 35.289998237 MB/sec (without seq num) 2024/11/24-04:21:35.301915 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291753: 921735 keys, 36362842 bytes, temperature: kWarm 2024/11/24-04:21:35.302003 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393295301951, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291753, "file_size": 36362842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4436635323, "table_properties": {"data_size": 30293856, "index_size": 1753408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25808580, "raw_average_key_size": 28, "raw_value_size": 108409286, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921735, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393292, "oldest_key_time": 0, "file_creation_time": 1732393292, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291753, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:35.338992 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst) /dev/shm/ToplingTemp/Topling-267937-Q98sEl 2024/11/24-04:21:36.229045 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst): first pass time = 0.89's, 150.804'MB/sec 2024/11/24-04:21:37.616930 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst): old prealloc_size = 147639500, real_size = 30532976 2024/11/24-04:21:37.617523 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928187 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.60 avg-zval = 30.73 2024/11/24-04:21:37.618295 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:38.435948 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291754.sst): second pass time = 1.39's, 78.045'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.501'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 274.858'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35952.384'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.045'MB/sec zip pipeline throughput = 78.045'MB/sec entries = 928187 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 30.73 usrkeys = 928187 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 30.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928187 bytes = 3132640 } seq expand size = 6497309 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4454 value = 3.7944 dict = 1.00 all = 3.8884 } Zip/UnZip{ index = 0.0957 value = 0.2635 dict = 1.00 all = 0.2572 } ---------------------------- total value len = 2252.931535 GB avg = 0.199 KB (by entry num) total key len = 307.247079 GB avg = 0.027 KB total ukey len = 237.614811 GB avg = 0.015 KB total ukey num = 15.751665331 Billion total entry num = 11.292979279 Billion write speed all = 36.581180796 MB/sec (with seq num) write speed all = 35.290300497 MB/sec (without seq num) 2024/11/24-04:21:38.480902 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291754: 928187 keys, 34517306 bytes, temperature: kWarm 2024/11/24-04:21:38.480970 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393298480934, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291754, "file_size": 34517306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4363142007, "largest_seqno": 4436634410, "table_properties": {"data_size": 28523552, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989236, "raw_average_key_size": 28, "raw_value_size": 108228542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928187, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393295, "oldest_key_time": 0, "file_creation_time": 1732393295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:38.517471 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst) /dev/shm/ToplingTemp/Topling-267937-FPXqEm 2024/11/24-04:21:39.612433 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst): first pass time = 1.09's, 122.581'MB/sec 2024/11/24-04:21:40.938969 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst): old prealloc_size = 147639500, real_size = 32329232 2024/11/24-04:21:40.939540 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922240 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.53 avg-zval = 32.90 2024/11/24-04:21:40.940339 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:41.772132 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291755.sst): second pass time = 1.33's, 81.758'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.709'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3400.891'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 288.633'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32145.405'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.758'MB/sec zip pipeline throughput = 81.758'MB/sec entries = 922240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.53 avg-zval = 32.90 usrkeys = 922240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.53 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922240 bytes = 3227856 } seq expand size = 6455680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5129 value = 3.5722 dict = 1.00 all = 3.6857 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2713 } ---------------------------- total value len = 2253.046386 GB avg = 0.199 KB (by entry num) total key len = 307.272902 GB avg = 0.027 KB total ukey len = 237.633255 GB avg = 0.015 KB total ukey num = 15.752587571 Billion total entry num = 11.293901519 Billion write speed all = 36.581447008 MB/sec (with seq num) write speed all = 35.290522827 MB/sec (without seq num) 2024/11/24-04:21:41.816567 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291755: 922240 keys, 36415636 bytes, temperature: kWarm 2024/11/24-04:21:41.816638 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393301816602, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291755, "file_size": 36415636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4436634614, "table_properties": {"data_size": 30344000, "index_size": 1754496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25822720, "raw_average_key_size": 28, "raw_value_size": 108395128, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393298, "oldest_key_time": 0, "file_creation_time": 1732393298, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:41.852576 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst) /dev/shm/ToplingTemp/Topling-267937-CQACdn 2024/11/24-04:21:42.903275 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst): first pass time = 1.05's, 127.746'MB/sec 2024/11/24-04:21:44.215494 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst): old prealloc_size = 147639500, real_size = 30648832 2024/11/24-04:21:44.216082 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927715 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.68 avg-zval = 30.87 2024/11/24-04:21:44.217007 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:45.062851 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291756.sst): second pass time = 1.31's, 82.534'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.872'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.027'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 266.391'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36238.846'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.534'MB/sec zip pipeline throughput = 82.534'MB/sec entries = 927715 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.68 avg-zval = 30.87 usrkeys = 927715 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.68 avg-zval = 30.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927715 bytes = 3131056 } seq expand size = 6494005 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4454 value = 3.7793 dict = 1.00 all = 3.8755 } Zip/UnZip{ index = 0.0957 value = 0.2646 dict = 1.00 all = 0.2580 } ---------------------------- total value len = 2253.161122 GB avg = 0.199 KB (by entry num) total key len = 307.298878 GB avg = 0.027 KB total ukey len = 237.651810 GB avg = 0.015 KB total ukey num = 15.753515286 Billion total entry num = 11.294829234 Billion write speed all = 36.581737501 MB/sec (with seq num) write speed all = 35.290767978 MB/sec (without seq num) 2024/11/24-04:21:45.105423 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291756: 927715 keys, 34632168 bytes, temperature: kWarm 2024/11/24-04:21:45.105491 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393305105457, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291756, "file_size": 34632168, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4360939150, "largest_seqno": 4436634421, "table_properties": {"data_size": 28640368, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976020, "raw_average_key_size": 28, "raw_value_size": 108241810, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927715, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393301, "oldest_key_time": 0, "file_creation_time": 1732393301, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:45.142393 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst) /dev/shm/ToplingTemp/Topling-267937-u1SnHj 2024/11/24-04:21:46.247613 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst): first pass time = 1.11's, 121.444'MB/sec 2024/11/24-04:21:47.602597 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst): old prealloc_size = 147639500, real_size = 32512528 2024/11/24-04:21:47.603155 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922848 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.44 avg-zval = 33.08 2024/11/24-04:21:47.605459 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:48.448057 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291757.sst): second pass time = 1.35's, 80.038'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.292'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.781'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 282.427'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34762.844'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.038'MB/sec zip pipeline throughput = 80.038'MB/sec entries = 922848 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 33.08 usrkeys = 922848 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.44 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922848 bytes = 3229984 } seq expand size = 6459936 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5137 value = 3.5503 dict = 1.00 all = 3.6671 } Zip/UnZip{ index = 0.0951 value = 0.2817 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 2253.275960 GB avg = 0.199 KB (by entry num) total key len = 307.324718 GB avg = 0.027 KB total ukey len = 237.670267 GB avg = 0.015 KB total ukey num = 15.754438134 Billion total entry num = 11.295752082 Billion write speed all = 36.581978100 MB/sec (with seq num) write speed all = 35.290965538 MB/sec (without seq num) 2024/11/24-04:21:48.493455 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291757: 922848 keys, 36600302 bytes, temperature: kWarm 2024/11/24-04:21:48.493529 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393308493493, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291757, "file_size": 36600302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4436635058, "table_properties": {"data_size": 30526144, "index_size": 1755520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839744, "raw_average_key_size": 28, "raw_value_size": 108378016, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393305, "oldest_key_time": 0, "file_creation_time": 1732393305, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:48.534859 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst) /dev/shm/ToplingTemp/Topling-267937-VKmqBl 2024/11/24-04:21:49.584717 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst): first pass time = 1.05's, 127.849'MB/sec 2024/11/24-04:21:50.915209 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst): old prealloc_size = 147639500, real_size = 30663904 2024/11/24-04:21:50.915780 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927111 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.77 avg-zval = 30.90 2024/11/24-04:21:50.916579 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:51.736725 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291758.sst): second pass time = 1.33's, 81.412'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.174'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.324'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 271.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35809.855'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.412'MB/sec zip pipeline throughput = 81.412'MB/sec entries = 927111 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.77 avg-zval = 30.90 usrkeys = 927111 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.77 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927111 bytes = 3129008 } seq expand size = 6489777 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3761 value = 3.7793 dict = 1.00 all = 3.8740 } Zip/UnZip{ index = 0.0964 value = 0.2646 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 2253.390708 GB avg = 0.199 KB (by entry num) total key len = 307.350677 GB avg = 0.027 KB total ukey len = 237.688809 GB avg = 0.015 KB total ukey num = 15.755365245 Billion total entry num = 11.296679193 Billion write speed all = 36.582269555 MB/sec (with seq num) write speed all = 35.291211690 MB/sec (without seq num) 2024/11/24-04:21:51.780676 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291758: 927111 keys, 34645642 bytes, temperature: kWarm 2024/11/24-04:21:51.780745 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393311780710, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291758, "file_size": 34645642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4358257058, "largest_seqno": 4436634436, "table_properties": {"data_size": 28644944, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959108, "raw_average_key_size": 28, "raw_value_size": 108258710, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927111, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393308, "oldest_key_time": 0, "file_creation_time": 1732393308, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:51.818068 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst) /dev/shm/ToplingTemp/Topling-267937-fxoM1i 2024/11/24-04:21:52.759451 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst): first pass time = 0.94's, 142.580'MB/sec 2024/11/24-04:21:54.091143 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst): old prealloc_size = 147639500, real_size = 32616608 2024/11/24-04:21:54.091692 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923487 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.34 avg-zval = 33.17 2024/11/24-04:21:54.093717 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:54.927182 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291759.sst): second pass time = 1.33's, 81.415'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.906'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.984'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 287.845'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29530.189'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.415'MB/sec zip pipeline throughput = 81.415'MB/sec entries = 923487 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 33.17 usrkeys = 923487 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923487 bytes = 3232224 } seq expand size = 6464409 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5129 value = 3.5379 dict = 1.00 all = 3.6565 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 2253.505533 GB avg = 0.199 KB (by entry num) total key len = 307.376534 GB avg = 0.027 KB total ukey len = 237.707279 GB avg = 0.015 KB total ukey num = 15.756288732 Billion total entry num = 11.297602680 Billion write speed all = 36.582611942 MB/sec (with seq num) write speed all = 35.291507381 MB/sec (without seq num) 2024/11/24-04:21:54.972228 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291759: 923487 keys, 36706130 bytes, temperature: kWarm 2024/11/24-04:21:54.972303 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393314972264, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291759, "file_size": 36706130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4436633217, "table_properties": {"data_size": 30628688, "index_size": 1756864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857636, "raw_average_key_size": 28, "raw_value_size": 108360118, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923487, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393311, "oldest_key_time": 0, "file_creation_time": 1732393311, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:55.009071 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst) /dev/shm/ToplingTemp/Topling-267937-gbQKGj 2024/11/24-04:21:55.842753 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst): first pass time = 0.83's, 161.001'MB/sec 2024/11/24-04:21:57.149701 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst): old prealloc_size = 147639500, real_size = 30814864 2024/11/24-04:21:57.150330 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926485 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.87 avg-zval = 31.10 2024/11/24-04:21:57.151108 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:21:57.969827 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291760.sst): second pass time = 1.31's, 82.899'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.427'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2926.819'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 273.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32474.591'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.899'MB/sec zip pipeline throughput = 82.899'MB/sec entries = 926485 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 31.10 usrkeys = 926485 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.87 avg-zval = 31.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926485 bytes = 3126896 } seq expand size = 6485395 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4469 value = 3.7584 dict = 1.00 all = 3.8574 } Zip/UnZip{ index = 0.0957 value = 0.2661 dict = 1.00 all = 0.2592 } ---------------------------- total value len = 2253.620294 GB avg = 0.199 KB (by entry num) total key len = 307.402476 GB avg = 0.027 KB total ukey len = 237.725808 GB avg = 0.015 KB total ukey num = 15.757215217 Billion total entry num = 11.298529165 Billion write speed all = 36.583031841 MB/sec (with seq num) write speed all = 35.291877520 MB/sec (without seq num) 2024/11/24-04:21:58.013781 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291760: 926485 keys, 34794656 bytes, temperature: kWarm 2024/11/24-04:21:58.013854 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393318013817, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291760, "file_size": 34794656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4355598202, "largest_seqno": 4436460235, "table_properties": {"data_size": 28809344, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941580, "raw_average_key_size": 28, "raw_value_size": 108276254, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393314, "oldest_key_time": 0, "file_creation_time": 1732393314, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:58.058607 139957466945088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst) /dev/shm/ToplingTemp/Topling-267937-SPbrrj 2024/11/24-04:21:58.138403 139957466945088 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst): first pass time = 0.08's, 240.317'MB/sec 2024/11/24-04:21:58.315241 139957466945088 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst): old prealloc_size = 147639500, real_size = 5221184 2024/11/24-04:21:58.315376 139957466945088 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 134981 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0154 GB zip-val = 0.0049 GB avg-val = 114.00 avg-zval = 36.50 2024/11/24-04:21:58.315518 139957466945088 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/24-04:21:58.440021 139957466945088 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00/291761.sst): second pass time = 0.18's, 87.349'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 46.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2326.304'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.320'MB/sec write SST data time = 0.02's, 299.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.161'MB, throughput = 5834.483'MB/sec zip my value time = 0.18's, unzip length = 0.015'GB zip my value throughput = 87.349'MB/sec zip pipeline throughput = 87.349'MB/sec entries = 134981 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 36.50 usrkeys = 134981 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 36.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 134981 bytes = 404960 } seq expand size = 944867 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0027 GB value = 0.0154 GB dict = 0.16 MB all = 0.0192 GB } __ZipSize{ index = 0.0003 GB value = 0.0049 GB dict = 0.16 MB all = 0.0058 GB } UnZip/Zip{ index = 10.3921 value = 3.1229 dict = 1.00 all = 3.3308 } Zip/UnZip{ index = 0.0962 value = 0.3202 dict = 1.00 all = 0.3002 } ---------------------------- total value len = 2253.636627 GB avg = 0.199 KB (by entry num) total key len = 307.406255 GB avg = 0.027 KB total ukey len = 237.728508 GB avg = 0.015 KB total ukey num = 15.757350198 Billion total entry num = 11.298664146 Billion write speed all = 36.583073426 MB/sec (with seq num) write speed all = 35.291912351 MB/sec (without seq num) 2024/11/24-04:21:58.452961 139957466945088 [compaction_job.cc:1948] [default] [JOB 41422] Generated table #291761: 134981 keys, 5754627 bytes, temperature: kWarm 2024/11/24-04:21:58.453044 139957466945088 EVENT_LOG_v1 {"time_micros": 1732393318453013, "cf_name": "default", "job": 41422, "event": "table_file_creation", "file_number": 291761, "file_size": 5754627, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4416786187, "largest_seqno": 4431650043, "table_properties": {"data_size": 4927472, "index_size": 259776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3779468, "raw_average_key_size": 28, "raw_value_size": 15387834, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 134981, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732393318, "oldest_key_time": 0, "file_creation_time": 1732393318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291761, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:21:58.462753 139957466945088 [compaction_job.cc:703] job-41422: subcompact[0], size: 0.711789 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41422/att-00 [291741,291742,291743,291744,291745,291746,291747,291748,291749,291750,291751,291752,291753,291754,291755,291756,291757,291758,291759,291760,291761] 2024/11/24-04:21:58.941911 139957466945088 (Original Log Time 2024/11/24-04:20:49.284606) EVENT_LOG_v1 {"time_micros": 1732393249284566, "job": 41422, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [291737], "files_L3": [291680, 291681, 291682, 291683, 291684, 291685, 291686, 291687, 291688, 291689, 291690, 291691, 291692, 291693, 291694, 291695, 291696, 291697, 291698, 291699], "score": 0.993674, "input_data_size": 728679156, "oldest_snapshot_seqno": 4436649546} 2024/11/24-04:21:58.941915 139957466945088 (Original Log Time 2024/11/24-04:21:58.938540) [compaction_job.cc:2022] [default] [JOB 41422] Compacted 1@2 + 20@3 files to L3 => 711789443 bytes 2024/11/24-04:21:58.941916 139957466945088 (Original Log Time 2024/11/24-04:21:58.941840) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 10.5 rd, 10.3 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(27.1, 667.8 +0.0 blob) out(678.8 +0.0 blob), read-write-amplify(50.7) write-amplify(25.1) OK, records in: 18789949, records dropped: 100914 output_compression: Snappy 2024/11/24-04:21:58.941920 139957466945088 (Original Log Time 2024/11/24-04:21:58.941885) EVENT_LOG_v1 {"time_micros": 1732393318941864, "cf": "default", "job": 41422, "event": "compaction_finished", "compaction_time_micros": 69178156, "compaction_time_cpu_micros": 29223177, "output_level": 3, "num_output_files": 21, "total_output_size": 711789443, "num_input_records": 18789949, "num_output_records": 18689035, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 525830540, "file_range_sync_nanos": 0, "file_fsync_nanos": 16586, "file_prepare_write_nanos": 49325, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/24-04:21:59.078709 139957466945088 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732393249,"executesMs":69778,"compactionJobId":41422,"attempt":0,"compactionInputRawBytes":2717853474,"compactionInputZipBytes":728679156,"compactionOutputRawBytes":0,"compactionOutputZipBytes":711789443,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4112ba3228f6933b444fe5bcb5099e74-e732fd6606c30059-00"}}, response = 2024/11/24-04:21:59.078798 139957466945088 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41422/att-00: olev 3, work 69.779 s, result 7.774 ms, install 3.395 ms, input{raw 2.531 GiB zip 694.923 MiB}