2024/11/23-14:36:02.477666 139957383018048 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00: fork to child time = 0.007660 sec 2024/11/23-14:36:02.478424 139957383018048 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00 2024/11/23-14:36:02.485770 139957383018048 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00 2024/11/23-14:36:02.486672 139957383018048 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:36:02.486920 139957383018048 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:36:02.541237 139957383018048 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28041/att-00/MANIFEST-206347 2024/11/23-14:36:02.574701 139957383018048 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28041/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 263763, last_sequence is 4029595689, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263628 2024/11/23-14:36:02.574709 139957383018048 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:36:02.574865 139957383018048 [version_set.cc:5721] Creating manifest 263763 2024/11/23-14:36:02.611013 139957383018048 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:36:02.633444 139957383018048 [compaction_job.cc:2437] [default] [JOB 28041] Compacting 1@2 + 28@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:36:02.633464 139957383018048 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20950 Base level 2, inputs: [263706(27MB)], [263727(40MB) 263728(2820KB) 263657(34MB) 263658(36MB) 263659(35MB) 263660(35MB) 263661(34MB) 263662(33MB) 263663(33MB) 263664(34MB) 263665(36MB) 263666(35MB) 263667(35MB) 263668(35MB) 263669(36MB) 263670(34MB) 263671(36MB) 263672(38MB) 263673(35MB) 263674(34MB) 263675(37MB) 263676(33MB) 263677(37MB) 263678(35MB) 263679(34MB) 263680(36MB) 263681(39MB) 263760(40MB)] 2024/11/23-14:36:02.640045 139957383018048 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:36:02.640073 139957383018048 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:36:02.640092 139957383018048 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:36:02.640097 139957383018048 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:36:02.640108 139957383018048 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:36:03.365098 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst) /dev/shm/ToplingTemp/Topling-76419-YuXENE 2024/11/23-14:36:03.917167 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst): first pass time = 0.55's, 243.137'MB/sec 2024/11/23-14:36:05.292085 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst): old prealloc_size = 147639500, real_size = 38479792 2024/11/23-14:36:05.292631 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924654 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0365 GB avg-val = 117.15 avg-zval = 39.46 2024/11/23-14:36:05.293411 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:06.084617 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263764.sst): second pass time = 1.37's, 78.841'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 53.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.619'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.349'MB/sec write SST data time = 0.13's, 328.669'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35695.954'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.841'MB/sec zip pipeline throughput = 78.841'MB/sec entries = 924654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 39.46 usrkeys = 924654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 39.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924654 bytes = 3005136 } seq expand size = 6472578 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0365 GB dict = 1.08 MB all = 0.0423 GB } UnZip/Zip{ index = 10.5132 value = 2.9687 dict = 1.00 all = 3.1696 } Zip/UnZip{ index = 0.0951 value = 0.3368 dict = 1.00 all = 0.3155 } ---------------------------- total value len = 8.963420 GB avg = 0.168 KB (by entry num) total key len = 1.466711 GB avg = 0.028 KB total ukey len = 1.016371 GB avg = 0.015 KB total ukey num = 0.066475089 Billion total entry num = 0.053282818 Billion write speed all = 0.509934368 MB/sec (with seq num) write speed all = 0.489094178 MB/sec (without seq num) 2024/11/23-14:36:06.135592 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263764: 924654 keys, 42344828 bytes, temperature: kWarm 2024/11/23-14:36:06.135704 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343766135632, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263764, "file_size": 42344828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3967285072, "largest_seqno": 4028963382, "table_properties": {"data_size": 36489376, "index_size": 1759040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890312, "raw_average_key_size": 28, "raw_value_size": 108327492, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343763, "oldest_key_time": 0, "file_creation_time": 1732343763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263764, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:06.186569 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst) /dev/shm/ToplingTemp/Topling-76419-BPlvWB 2024/11/23-14:36:07.097704 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst): first pass time = 0.91's, 147.315'MB/sec 2024/11/23-14:36:08.389059 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst): old prealloc_size = 147639500, real_size = 31863344 2024/11/23-14:36:08.389608 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935004 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0299 GB avg-val = 115.55 avg-zval = 31.93 2024/11/23-14:36:08.390464 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:09.244646 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263765.sst): second pass time = 1.29's, 83.719'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.198'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3427.893'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.322'MB/sec write SST data time = 0.13's, 280.036'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36917.740'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.719'MB/sec zip pipeline throughput = 83.719'MB/sec entries = 935004 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.55 avg-zval = 31.93 usrkeys = 935004 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.55 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935004 bytes = 3389408 } seq expand size = 6545028 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5127 value = 3.6193 dict = 1.00 all = 3.7176 } Zip/UnZip{ index = 0.0951 value = 0.2763 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 9.078003 GB avg = 0.167 KB (by entry num) total key len = 1.492892 GB avg = 0.028 KB total ukey len = 1.035071 GB avg = 0.015 KB total ukey num = 0.067410093 Billion total entry num = 0.054217822 Billion write speed all = 0.516736499 MB/sec (with seq num) write speed all = 0.495533882 MB/sec (without seq num) 2024/11/23-14:36:09.288844 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263765: 935004 keys, 36102910 bytes, temperature: kWarm 2024/11/23-14:36:09.288916 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343769288880, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263765, "file_size": 36102910, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 4012776090, "table_properties": {"data_size": 29850592, "index_size": 1778816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26180112, "raw_average_key_size": 28, "raw_value_size": 108037660, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935004, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343766, "oldest_key_time": 0, "file_creation_time": 1732343766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263765, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:09.326904 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst) /dev/shm/ToplingTemp/Topling-76419-8vqicF 2024/11/23-14:36:10.287918 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst): first pass time = 0.96's, 139.668'MB/sec 2024/11/23-14:36:11.619530 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst): old prealloc_size = 147639500, real_size = 34219152 2024/11/23-14:36:11.620115 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924002 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 117.26 avg-zval = 34.87 2024/11/23-14:36:11.620922 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:12.471760 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263766.sst): second pass time = 1.33's, 81.415'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.089'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.710'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 299.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35897.363'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.415'MB/sec zip pipeline throughput = 81.415'MB/sec entries = 924002 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.26 avg-zval = 34.87 usrkeys = 924002 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.26 avg-zval = 34.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924002 bytes = 3349520 } seq expand size = 6468014 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4446 value = 3.3628 dict = 1.00 all = 3.4929 } Zip/UnZip{ index = 0.0957 value = 0.2974 dict = 1.00 all = 0.2863 } ---------------------------- total value len = 9.192816 GB avg = 0.167 KB (by entry num) total key len = 1.518764 GB avg = 0.028 KB total ukey len = 1.053551 GB avg = 0.015 KB total ukey num = 0.068334095 Billion total entry num = 0.055141824 Billion write speed all = 0.523531049 MB/sec (with seq num) write speed all = 0.501970490 MB/sec (without seq num) 2024/11/23-14:36:12.518121 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263766: 924002 keys, 38425566 bytes, temperature: kWarm 2024/11/23-14:36:12.518187 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343772518153, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263766, "file_size": 38425566, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4012776108, "table_properties": {"data_size": 32218624, "index_size": 1769344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25872056, "raw_average_key_size": 28, "raw_value_size": 108345748, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924002, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343769, "oldest_key_time": 0, "file_creation_time": 1732343769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263766, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:12.556457 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst) /dev/shm/ToplingTemp/Topling-76419-CB8LiF 2024/11/23-14:36:13.526512 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst): first pass time = 0.97's, 138.366'MB/sec 2024/11/23-14:36:14.845291 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst): old prealloc_size = 147639500, real_size = 32467472 2024/11/23-14:36:14.845860 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931642 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.07 avg-zval = 32.67 2024/11/23-14:36:14.846690 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:15.700616 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263767.sst): second pass time = 1.32's, 82.070'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.904'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.571'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 285.867'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35516.276'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.070'MB/sec zip pipeline throughput = 82.070'MB/sec entries = 931642 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.07 avg-zval = 32.67 usrkeys = 931642 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.07 avg-zval = 32.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931642 bytes = 3377216 } seq expand size = 6521494 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.3778 value = 3.5524 dict = 1.00 all = 3.6576 } Zip/UnZip{ index = 0.0964 value = 0.2815 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 9.307470 GB avg = 0.166 KB (by entry num) total key len = 1.544850 GB avg = 0.028 KB total ukey len = 1.072184 GB avg = 0.015 KB total ukey num = 0.069265737 Billion total entry num = 0.056073466 Billion write speed all = 0.530326024 MB/sec (with seq num) write speed all = 0.508404650 MB/sec (without seq num) 2024/11/23-14:36:15.745213 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263767: 931642 keys, 36695670 bytes, temperature: kWarm 2024/11/23-14:36:15.745282 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343775745246, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263767, "file_size": 36695670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4012776122, "table_properties": {"data_size": 30438912, "index_size": 1795456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085976, "raw_average_key_size": 28, "raw_value_size": 108131812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343772, "oldest_key_time": 0, "file_creation_time": 1732343772, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263767, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:15.780611 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst) /dev/shm/ToplingTemp/Topling-76419-xMthiD 2024/11/23-14:36:16.806992 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst): first pass time = 1.03's, 130.772'MB/sec 2024/11/23-14:36:18.121166 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst): old prealloc_size = 147639500, real_size = 32574528 2024/11/23-14:36:18.121741 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931352 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.11 avg-zval = 32.81 2024/11/23-14:36:18.122574 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:18.981806 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263768.sst): second pass time = 1.31's, 82.364'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.898'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3225.804'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 284.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29465.337'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.364'MB/sec zip pipeline throughput = 82.364'MB/sec entries = 931352 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.11 avg-zval = 32.81 usrkeys = 931352 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.11 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931352 bytes = 3492592 } seq expand size = 6519464 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4472 value = 3.5388 dict = 1.00 all = 3.6355 } Zip/UnZip{ index = 0.0957 value = 0.2826 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 9.422129 GB avg = 0.165 KB (by entry num) total key len = 1.570927 GB avg = 0.028 KB total ukey len = 1.090811 GB avg = 0.016 KB total ukey num = 0.070197089 Billion total entry num = 0.057004818 Billion write speed all = 0.537117385 MB/sec (with seq num) write speed all = 0.514835481 MB/sec (without seq num) 2024/11/23-14:36:19.028251 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263768: 931352 keys, 36918300 bytes, temperature: kWarm 2024/11/23-14:36:19.028320 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343779028284, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263768, "file_size": 36918300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 30558512, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077856, "raw_average_key_size": 28, "raw_value_size": 108139968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343775, "oldest_key_time": 0, "file_creation_time": 1732343775, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263768, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:19.064722 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst) /dev/shm/ToplingTemp/Topling-76419-W4G9nD 2024/11/23-14:36:19.957633 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst): first pass time = 0.89's, 150.322'MB/sec 2024/11/23-14:36:21.256266 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst): old prealloc_size = 147639500, real_size = 32558880 2024/11/23-14:36:21.256815 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932719 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.90 avg-zval = 32.76 2024/11/23-14:36:21.257634 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:22.120138 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263769.sst): second pass time = 1.30's, 83.299'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.588'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.532'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 286.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35402.917'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.299'MB/sec zip pipeline throughput = 83.299'MB/sec entries = 932719 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.90 avg-zval = 32.76 usrkeys = 932719 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.90 avg-zval = 32.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932719 bytes = 3381120 } seq expand size = 6529033 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5146 value = 3.5384 dict = 1.00 all = 3.6482 } Zip/UnZip{ index = 0.0951 value = 0.2826 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 9.536760 GB avg = 0.165 KB (by entry num) total key len = 1.597044 GB avg = 0.028 KB total ukey len = 1.109465 GB avg = 0.016 KB total ukey num = 0.071129808 Billion total entry num = 0.057937537 Billion write speed all = 0.543910831 MB/sec (with seq num) write speed all = 0.521267820 MB/sec (without seq num) 2024/11/23-14:36:22.170207 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263769: 932719 keys, 36790250 bytes, temperature: kWarm 2024/11/23-14:36:22.170275 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343782170239, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263769, "file_size": 36790250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 30551376, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26116132, "raw_average_key_size": 28, "raw_value_size": 108101702, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932719, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343779, "oldest_key_time": 0, "file_creation_time": 1732343779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263769, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:22.212439 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst) /dev/shm/ToplingTemp/Topling-76419-DpDN3D 2024/11/23-14:36:23.229327 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst): first pass time = 1.02's, 131.994'MB/sec 2024/11/23-14:36:24.528486 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst): old prealloc_size = 147639500, real_size = 32047152 2024/11/23-14:36:24.529066 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934763 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0300 GB avg-val = 115.58 avg-zval = 32.13 2024/11/23-14:36:24.529865 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:25.382485 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263770.sst): second pass time = 1.30's, 83.243'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.354'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.737'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 288.673'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35141.702'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.243'MB/sec zip pipeline throughput = 83.243'MB/sec entries = 934763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.58 avg-zval = 32.13 usrkeys = 934763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 32.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934763 bytes = 3271680 } seq expand size = 6543341 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5122 value = 3.5973 dict = 1.00 all = 3.7108 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2695 } ---------------------------- total value len = 9.651348 GB avg = 0.164 KB (by entry num) total key len = 1.623217 GB avg = 0.028 KB total ukey len = 1.128161 GB avg = 0.016 KB total ukey num = 0.072064571 Billion total entry num = 0.058872300 Billion write speed all = 0.550699556 MB/sec (with seq num) write speed all = 0.527694890 MB/sec (without seq num) 2024/11/23-14:36:25.429820 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263770: 934763 keys, 36169130 bytes, temperature: kWarm 2024/11/23-14:36:25.429893 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343785429855, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263770, "file_size": 36169130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30034848, "index_size": 1778432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173364, "raw_average_key_size": 28, "raw_value_size": 108044426, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343782, "oldest_key_time": 0, "file_creation_time": 1732343782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263770, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:25.467733 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst) /dev/shm/ToplingTemp/Topling-76419-RwxzkF 2024/11/23-14:36:26.412273 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst): first pass time = 0.94's, 142.104'MB/sec 2024/11/23-14:36:27.714480 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst): old prealloc_size = 147639500, real_size = 31589184 2024/11/23-14:36:27.715058 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.03 avg-zval = 31.75 2024/11/23-14:36:27.715852 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:28.555674 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263771.sst): second pass time = 1.30's, 83.115'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3195.333'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.12's, 287.108'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33848.560'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.115'MB/sec zip pipeline throughput = 83.115'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 31.75 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 31.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5147 value = 3.6549 dict = 1.00 all = 3.7595 } Zip/UnZip{ index = 0.0951 value = 0.2736 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 9.765996 GB avg = 0.163 KB (by entry num) total key len = 1.649310 GB avg = 0.028 KB total ukey len = 1.146798 GB avg = 0.016 KB total ukey num = 0.072996456 Billion total entry num = 0.059804185 Billion write speed all = 0.557487565 MB/sec (with seq num) write speed all = 0.534122381 MB/sec (without seq num) 2024/11/23-14:36:28.600538 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263771: 931885 keys, 35701404 bytes, temperature: kWarm 2024/11/23-14:36:28.600607 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343788600572, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263771, "file_size": 35701404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 29583472, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343785, "oldest_key_time": 0, "file_creation_time": 1732343785, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263771, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:28.637664 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst) /dev/shm/ToplingTemp/Topling-76419-AR2ekG 2024/11/23-14:36:29.677719 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst): first pass time = 1.04's, 129.053'MB/sec 2024/11/23-14:36:31.036159 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst): old prealloc_size = 147639500, real_size = 35020752 2024/11/23-14:36:31.036813 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083732 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 95.85 avg-zval = 30.14 2024/11/23-14:36:31.037053 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:31.727189 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263772.sst): second pass time = 1.36's, 76.536'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 52.564'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.874'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.489'MB/sec write SST data time = 0.14's, 266.317'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.031'MB, throughput = 34490.483'MB/sec zip my value time = 1.36's, unzip length = 0.104'GB zip my value throughput = 76.536'MB/sec zip pipeline throughput = 76.536'MB/sec entries = 1083732 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.85 avg-zval = 30.14 usrkeys = 1083732 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.59 avg-zval = 30.14 TagRS{ kinds = RS_Key0_TagN bytes = 169440 } TagArray{ size = 115540 bytes = 361072 } seq expand size = 808780 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3793 value = 3.1803 dict = 1.00 all = 3.6952 } Zip/UnZip{ index = 0.0963 value = 0.3144 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 9.870678 GB avg = 0.162 KB (by entry num) total key len = 1.679654 GB avg = 0.028 KB total ukey len = 1.168473 GB avg = 0.016 KB total ukey num = 0.074080188 Billion total entry num = 0.060887917 Billion write speed all = 0.563994485 MB/sec (with seq num) write speed all = 0.540209576 MB/sec (without seq num) 2024/11/23-14:36:31.772893 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263772: 1083732 keys, 36322367 bytes, temperature: kWarm 2024/11/23-14:36:31.772963 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343791772928, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263772, "file_size": 36322367, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32661376, "index_size": 2088256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30344496, "raw_average_key_size": 28, "raw_value_size": 103873365, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343788, "oldest_key_time": 0, "file_creation_time": 1732343788, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263772, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:31.810048 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst) /dev/shm/ToplingTemp/Topling-76419-9uGAuD 2024/11/23-14:36:32.828928 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst): first pass time = 1.02's, 131.736'MB/sec 2024/11/23-14:36:34.175612 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst): old prealloc_size = 147639500, real_size = 35516112 2024/11/23-14:36:34.176173 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922855 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0335 GB avg-val = 117.44 avg-zval = 36.33 2024/11/23-14:36:34.176688 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:34.952606 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263773.sst): second pass time = 1.35's, 80.524'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 49.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.475'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.353'MB/sec write SST data time = 0.12's, 313.286'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31512.180'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.524'MB/sec zip pipeline throughput = 80.524'MB/sec entries = 922855 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 36.33 usrkeys = 922855 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.25 avg-zval = 36.33 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 502644 bytes = 1884928 } seq expand size = 3518508 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5122 value = 3.2323 dict = 1.00 all = 3.4946 } Zip/UnZip{ index = 0.0951 value = 0.3094 dict = 1.00 all = 0.2862 } ---------------------------- total value len = 9.982574 GB avg = 0.162 KB (by entry num) total key len = 1.705494 GB avg = 0.028 KB total ukey len = 1.186930 GB avg = 0.016 KB total ukey num = 0.075003043 Billion total entry num = 0.061810772 Billion write speed all = 0.570630180 MB/sec (with seq num) write speed all = 0.546488574 MB/sec (without seq num) 2024/11/23-14:36:35.001234 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263773: 922855 keys, 38406901 bytes, temperature: kWarm 2024/11/23-14:36:35.001306 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343795001269, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263773, "file_size": 38406901, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3984016719, "table_properties": {"data_size": 33529408, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839940, "raw_average_key_size": 28, "raw_value_size": 108377858, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343791, "oldest_key_time": 0, "file_creation_time": 1732343791, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263773, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:35.044046 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst) /dev/shm/ToplingTemp/Topling-76419-RVkRNC 2024/11/23-14:36:35.881239 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst): first pass time = 0.84's, 160.326'MB/sec 2024/11/23-14:36:37.185815 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst): old prealloc_size = 147639500, real_size = 32677680 2024/11/23-14:36:37.186414 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.10 avg-zval = 32.93 2024/11/23-14:36:37.187231 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:38.038435 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263774.sst): second pass time = 1.30's, 82.943'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.286'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.182'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 293.960'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36772.799'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.943'MB/sec zip pipeline throughput = 82.943'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 32.93 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5148 value = 3.5255 dict = 1.00 all = 3.6368 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 10.097232 GB avg = 0.161 KB (by entry num) total key len = 1.731574 GB avg = 0.028 KB total ukey len = 1.205559 GB avg = 0.016 KB total ukey num = 0.075934472 Billion total entry num = 0.062742201 Billion write speed all = 0.577414232 MB/sec (with seq num) write speed all = 0.552912527 MB/sec (without seq num) 2024/11/23-14:36:38.084742 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263774: 931429 keys, 36905404 bytes, temperature: kWarm 2024/11/23-14:36:38.084813 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343798084777, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263774, "file_size": 36905404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30672992, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343795, "oldest_key_time": 0, "file_creation_time": 1732343795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263774, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:38.128076 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst) /dev/shm/ToplingTemp/Topling-76419-OUtLtE 2024/11/23-14:36:39.127861 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst): first pass time = 1.00's, 134.252'MB/sec 2024/11/23-14:36:40.433999 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst): old prealloc_size = 147639500, real_size = 32691792 2024/11/23-14:36:40.434556 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.12 avg-zval = 32.95 2024/11/23-14:36:40.435366 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:41.287609 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263775.sst): second pass time = 1.31's, 82.846'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.114'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.853'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 293.866'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32138.268'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.846'MB/sec zip pipeline throughput = 82.846'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.95 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5116 value = 3.5240 dict = 1.00 all = 3.6469 } Zip/UnZip{ index = 0.0951 value = 0.2838 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.211892 GB avg = 0.160 KB (by entry num) total key len = 1.757651 GB avg = 0.028 KB total ukey len = 1.224185 GB avg = 0.016 KB total ukey num = 0.076865781 Billion total entry num = 0.063673510 Billion write speed all = 0.584191545 MB/sec (with seq num) write speed all = 0.559330094 MB/sec (without seq num) 2024/11/23-14:36:41.333961 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263775: 931309 keys, 36802864 bytes, temperature: kWarm 2024/11/23-14:36:41.334040 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343801334002, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263775, "file_size": 36802864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30686784, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343798, "oldest_key_time": 0, "file_creation_time": 1732343798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263775, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:41.370471 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst) /dev/shm/ToplingTemp/Topling-76419-T12VkF 2024/11/23-14:36:42.235632 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst): first pass time = 0.87's, 155.142'MB/sec 2024/11/23-14:36:43.561082 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst): old prealloc_size = 147639500, real_size = 32740288 2024/11/23-14:36:43.561662 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.09 avg-zval = 32.98 2024/11/23-14:36:43.562467 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:44.416776 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263776.sst): second pass time = 1.32's, 81.668'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.340'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3190.418'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 296.006'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31355.221'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.668'MB/sec zip pipeline throughput = 81.668'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.09 avg-zval = 32.98 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.09 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4446 value = 3.5197 dict = 1.00 all = 3.6192 } Zip/UnZip{ index = 0.0957 value = 0.2841 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 10.326550 GB avg = 0.160 KB (by entry num) total key len = 1.783732 GB avg = 0.028 KB total ukey len = 1.242814 GB avg = 0.016 KB total ukey num = 0.077797243 Billion total entry num = 0.064604972 Billion write speed all = 0.590970221 MB/sec (with seq num) write speed all = 0.565748930 MB/sec (without seq num) 2024/11/23-14:36:44.464255 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263776: 931462 keys, 37084500 bytes, temperature: kWarm 2024/11/23-14:36:44.464321 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343804464284, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263776, "file_size": 37084500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30723632, "index_size": 1783616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343801, "oldest_key_time": 0, "file_creation_time": 1732343801, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263776, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:44.499744 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst) /dev/shm/ToplingTemp/Topling-76419-IK35SB 2024/11/23-14:36:45.430898 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst): first pass time = 0.93's, 144.147'MB/sec 2024/11/23-14:36:46.717411 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst): old prealloc_size = 147639500, real_size = 34358672 2024/11/23-14:36:46.717998 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.04 avg-zval = 34.98 2024/11/23-14:36:46.718838 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:47.559785 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263777.sst): second pass time = 1.29's, 84.244'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.787'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3078.331'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 309.497'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35050.013'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.244'MB/sec zip pipeline throughput = 84.244'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.98 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5133 value = 3.3462 dict = 1.00 all = 3.4800 } Zip/UnZip{ index = 0.0951 value = 0.2988 dict = 1.00 all = 0.2874 } ---------------------------- total value len = 10.441334 GB avg = 0.159 KB (by entry num) total key len = 1.809643 GB avg = 0.028 KB total ukey len = 1.261322 GB avg = 0.016 KB total ukey num = 0.078722648 Billion total entry num = 0.065530377 Billion write speed all = 0.597744352 MB/sec (with seq num) write speed all = 0.572165714 MB/sec (without seq num) 2024/11/23-14:36:47.606386 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263777: 925405 keys, 38568422 bytes, temperature: kWarm 2024/11/23-14:36:47.606455 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343807606419, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263777, "file_size": 38568422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32366720, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343804, "oldest_key_time": 0, "file_creation_time": 1732343804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263777, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:47.642168 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst) /dev/shm/ToplingTemp/Topling-76419-8KSimC 2024/11/23-14:36:48.473526 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst): first pass time = 0.83's, 161.453'MB/sec 2024/11/23-14:36:49.766783 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst): old prealloc_size = 147639500, real_size = 32529552 2024/11/23-14:36:49.767356 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925554 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.01 avg-zval = 32.98 2024/11/23-14:36:49.768172 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:50.603449 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263778.sst): second pass time = 1.29's, 83.795'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.913'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.093'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 292.045'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36370.266'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.795'MB/sec zip pipeline throughput = 83.795'MB/sec entries = 925554 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.01 avg-zval = 32.98 usrkeys = 925554 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.01 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925554 bytes = 3355152 } seq expand size = 6478878 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4447 value = 3.5479 dict = 1.00 all = 3.6532 } Zip/UnZip{ index = 0.0957 value = 0.2819 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 10.556115 GB avg = 0.159 KB (by entry num) total key len = 1.835559 GB avg = 0.028 KB total ukey len = 1.279833 GB avg = 0.016 KB total ukey num = 0.079648202 Billion total entry num = 0.066455931 Billion write speed all = 0.604519388 MB/sec (with seq num) write speed all = 0.578583328 MB/sec (without seq num) 2024/11/23-14:36:50.650181 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263778: 925554 keys, 36739458 bytes, temperature: kWarm 2024/11/23-14:36:50.650252 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343810650217, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263778, "file_size": 36739458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775987, "table_properties": {"data_size": 30525696, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915512, "raw_average_key_size": 28, "raw_value_size": 108302272, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925554, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343807, "oldest_key_time": 0, "file_creation_time": 1732343807, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263778, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:50.685732 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst) /dev/shm/ToplingTemp/Topling-76419-FigVtD 2024/11/23-14:36:51.738993 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst): first pass time = 1.05's, 127.437'MB/sec 2024/11/23-14:36:53.060494 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst): old prealloc_size = 147639500, real_size = 34115952 2024/11/23-14:36:53.061080 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929721 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.36 avg-zval = 34.54 2024/11/23-14:36:53.061905 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:53.912202 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263779.sst): second pass time = 1.32's, 81.941'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.015'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.176'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 301.946'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35530.047'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.941'MB/sec zip pipeline throughput = 81.941'MB/sec entries = 929721 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 34.54 usrkeys = 929721 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929721 bytes = 3370256 } seq expand size = 6508047 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5138 value = 3.3687 dict = 1.00 all = 3.5008 } Zip/UnZip{ index = 0.0951 value = 0.2968 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 10.670809 GB avg = 0.158 KB (by entry num) total key len = 1.861591 GB avg = 0.028 KB total ukey len = 1.298428 GB avg = 0.016 KB total ukey num = 0.080577923 Billion total entry num = 0.067385652 Billion write speed all = 0.611285935 MB/sec (with seq num) write speed all = 0.584991273 MB/sec (without seq num) 2024/11/23-14:36:53.958439 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263779: 929721 keys, 38338676 bytes, temperature: kWarm 2024/11/23-14:36:53.958514 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343813958475, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263779, "file_size": 38338676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 32114784, "index_size": 1768576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032188, "raw_average_key_size": 28, "raw_value_size": 108185670, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343810, "oldest_key_time": 0, "file_creation_time": 1732343810, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263779, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:53.996824 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst) /dev/shm/ToplingTemp/Topling-76419-dNcEdD 2024/11/23-14:36:54.997170 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst): first pass time = 1.00's, 134.178'MB/sec 2024/11/23-14:36:56.330860 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst): old prealloc_size = 147639500, real_size = 35717296 2024/11/23-14:36:56.331445 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930341 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0337 GB avg-val = 116.27 avg-zval = 36.23 2024/11/23-14:36:56.332216 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:36:57.144627 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263780.sst): second pass time = 1.33's, 81.151'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.567'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3120.365'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 299.433'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 38085.952'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.151'MB/sec zip pipeline throughput = 81.151'MB/sec entries = 930341 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.27 avg-zval = 36.23 usrkeys = 930341 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.89 avg-zval = 36.23 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747344 bytes = 2802560 } seq expand size = 5231408 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4467 value = 3.2094 dict = 1.00 all = 3.3961 } Zip/UnZip{ index = 0.0957 value = 0.3116 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 10.784208 GB avg = 0.158 KB (by entry num) total key len = 1.887640 GB avg = 0.028 KB total ukey len = 1.317035 GB avg = 0.016 KB total ukey num = 0.081508264 Billion total entry num = 0.068315993 Billion write speed all = 0.617990337 MB/sec (with seq num) write speed all = 0.591336847 MB/sec (without seq num) 2024/11/23-14:36:57.192488 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263780: 930341 keys, 39520774 bytes, temperature: kWarm 2024/11/23-14:36:57.192559 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343817192524, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263780, "file_size": 39520774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 33703392, "index_size": 1781120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26049548, "raw_average_key_size": 28, "raw_value_size": 108168266, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930341, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343813, "oldest_key_time": 0, "file_creation_time": 1732343813, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263780, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:36:57.226349 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst) /dev/shm/ToplingTemp/Topling-76419-kIXUrE 2024/11/23-14:36:58.263618 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst): first pass time = 1.04's, 129.401'MB/sec 2024/11/23-14:36:59.581966 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst): old prealloc_size = 147639500, real_size = 33017200 2024/11/23-14:36:59.582542 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.49 avg-zval = 33.38 2024/11/23-14:36:59.583380 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:00.436705 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263781.sst): second pass time = 1.32's, 82.128'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.568'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3401.741'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 294.178'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35573.970'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.128'MB/sec zip pipeline throughput = 82.128'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.38 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4452 value = 3.4899 dict = 1.00 all = 3.6043 } Zip/UnZip{ index = 0.0957 value = 0.2865 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 10.898919 GB avg = 0.157 KB (by entry num) total key len = 1.913650 GB avg = 0.028 KB total ukey len = 1.335613 GB avg = 0.016 KB total ukey num = 0.082437169 Billion total entry num = 0.069244898 Billion write speed all = 0.624752781 MB/sec (with seq num) write speed all = 0.597741215 MB/sec (without seq num) 2024/11/23-14:37:00.482173 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263781: 928905 keys, 37237830 bytes, temperature: kWarm 2024/11/23-14:37:00.482246 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343820482209, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263781, "file_size": 37237830, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31006176, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343817, "oldest_key_time": 0, "file_creation_time": 1732343817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263781, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:00.517348 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst) /dev/shm/ToplingTemp/Topling-76419-7LpAVB 2024/11/23-14:37:01.509632 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst): first pass time = 0.99's, 135.268'MB/sec 2024/11/23-14:37:02.801942 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst): old prealloc_size = 147639500, real_size = 32351840 2024/11/23-14:37:02.802509 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934575 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0303 GB avg-val = 115.61 avg-zval = 32.46 2024/11/23-14:37:02.803358 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:03.665612 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263782.sst): second pass time = 1.29's, 83.661'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.367'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.477'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 290.315'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35306.350'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.661'MB/sec zip pipeline throughput = 83.661'MB/sec entries = 934575 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.61 avg-zval = 32.46 usrkeys = 934575 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.61 avg-zval = 32.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934575 bytes = 3387856 } seq expand size = 6542025 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5127 value = 3.5613 dict = 1.00 all = 3.6681 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2726 } ---------------------------- total value len = 11.013511 GB avg = 0.157 KB (by entry num) total key len = 1.939818 GB avg = 0.028 KB total ukey len = 1.354304 GB avg = 0.016 KB total ukey num = 0.083371744 Billion total entry num = 0.070179473 Billion write speed all = 0.631516932 MB/sec (with seq num) write speed all = 0.604145111 MB/sec (without seq num) 2024/11/23-14:37:03.711162 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263782: 934575 keys, 36590102 bytes, temperature: kWarm 2024/11/23-14:37:03.711233 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343823711196, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263782, "file_size": 36590102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30340048, "index_size": 1777984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26168100, "raw_average_key_size": 28, "raw_value_size": 108049714, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934575, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343820, "oldest_key_time": 0, "file_creation_time": 1732343820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263782, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:03.758464 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst) /dev/shm/ToplingTemp/Topling-76419-dfyrYF 2024/11/23-14:37:04.619707 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst): first pass time = 0.86's, 155.849'MB/sec 2024/11/23-14:37:05.960126 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst): old prealloc_size = 147639500, real_size = 34802960 2024/11/23-14:37:05.960630 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931257 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0328 GB avg-val = 116.13 avg-zval = 35.21 2024/11/23-14:37:05.961487 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:06.799097 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263783.sst): second pass time = 1.34's, 80.727'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3713.950'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 310.797'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30570.003'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.727'MB/sec zip pipeline throughput = 80.727'MB/sec entries = 931257 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 35.21 usrkeys = 931257 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931257 bytes = 3492224 } seq expand size = 6518799 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.4435 value = 3.2984 dict = 1.00 all = 3.4286 } Zip/UnZip{ index = 0.0958 value = 0.3032 dict = 1.00 all = 0.2917 } ---------------------------- total value len = 11.128172 GB avg = 0.156 KB (by entry num) total key len = 1.965893 GB avg = 0.028 KB total ukey len = 1.372929 GB avg = 0.016 KB total ukey num = 0.084303001 Billion total entry num = 0.071110730 Billion write speed all = 0.638280790 MB/sec (with seq num) write speed all = 0.610549990 MB/sec (without seq num) 2024/11/23-14:37:06.847361 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263783: 931257 keys, 39146444 bytes, temperature: kWarm 2024/11/23-14:37:06.847435 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343826847399, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263783, "file_size": 39146444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32786560, "index_size": 1783424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075196, "raw_average_key_size": 28, "raw_value_size": 108142574, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931257, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343823, "oldest_key_time": 0, "file_creation_time": 1732343823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263783, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:06.888856 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst) /dev/shm/ToplingTemp/Topling-76419-qFnBOF 2024/11/23-14:37:07.875559 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst): first pass time = 0.99's, 136.032'MB/sec 2024/11/23-14:37:09.201866 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst): old prealloc_size = 147639500, real_size = 30266400 2024/11/23-14:37:09.202448 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930900 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.18 avg-zval = 30.34 2024/11/23-14:37:09.203443 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:10.023623 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263784.sst): second pass time = 1.33's, 81.593'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.870'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 273.688'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26386.757'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.593'MB/sec zip pipeline throughput = 81.593'MB/sec entries = 930900 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.18 avg-zval = 30.34 usrkeys = 930900 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.18 avg-zval = 30.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930900 bytes = 3490896 } seq expand size = 6516300 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3765 value = 3.8299 dict = 1.00 all = 3.8784 } Zip/UnZip{ index = 0.0964 value = 0.2611 dict = 1.00 all = 0.2578 } ---------------------------- total value len = 11.242841 GB avg = 0.156 KB (by entry num) total key len = 1.991958 GB avg = 0.028 KB total ukey len = 1.391547 GB avg = 0.016 KB total ukey num = 0.085233901 Billion total entry num = 0.072041630 Billion write speed all = 0.645039596 MB/sec (with seq num) write speed all = 0.616950191 MB/sec (without seq num) 2024/11/23-14:37:10.066806 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263784: 930900 keys, 34606902 bytes, temperature: kWarm 2024/11/23-14:37:10.066878 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343830066842, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263784, "file_size": 34606902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3430492379, "largest_seqno": 4012776300, "table_properties": {"data_size": 28239248, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065200, "raw_average_key_size": 28, "raw_value_size": 108152540, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930900, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343826, "oldest_key_time": 0, "file_creation_time": 1732343826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263784, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:10.102942 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst) /dev/shm/ToplingTemp/Topling-76419-2HMxVB 2024/11/23-14:37:11.174931 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst): first pass time = 1.07's, 125.216'MB/sec 2024/11/23-14:37:12.507544 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst): old prealloc_size = 147639500, real_size = 35211920 2024/11/23-14:37:12.508137 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923039 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.41 avg-zval = 36.00 2024/11/23-14:37:12.508948 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:13.349103 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263785.sst): second pass time = 1.33's, 81.402'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.975'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.853'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 316.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34648.431'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.402'MB/sec zip pipeline throughput = 81.402'MB/sec entries = 923039 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 36.00 usrkeys = 923039 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 36.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923039 bytes = 3461408 } seq expand size = 6461273 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5139 value = 3.2618 dict = 1.00 all = 3.3952 } Zip/UnZip{ index = 0.0951 value = 0.3066 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 11.357675 GB avg = 0.156 KB (by entry num) total key len = 2.017803 GB avg = 0.028 KB total ukey len = 1.410008 GB avg = 0.016 KB total ukey num = 0.086156940 Billion total entry num = 0.072964669 Billion write speed all = 0.651790393 MB/sec (with seq num) write speed all = 0.623345700 MB/sec (without seq num) 2024/11/23-14:37:13.396348 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263785: 923039 keys, 39531406 bytes, temperature: kWarm 2024/11/23-14:37:13.396418 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343833396382, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263785, "file_size": 39531406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33225152, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845092, "raw_average_key_size": 28, "raw_value_size": 108372714, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923039, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343830, "oldest_key_time": 0, "file_creation_time": 1732343830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263785, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:13.435784 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst) /dev/shm/ToplingTemp/Topling-76419-bePahF 2024/11/23-14:37:14.466701 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst): first pass time = 1.03's, 130.197'MB/sec 2024/11/23-14:37:15.755563 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst): old prealloc_size = 147639500, real_size = 31876336 2024/11/23-14:37:15.756158 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 116.15 avg-zval = 32.07 2024/11/23-14:37:15.757022 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:16.591567 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263786.sst): second pass time = 1.29's, 83.959'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3110.566'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 288.904'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34076.012'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.959'MB/sec zip pipeline throughput = 83.959'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.07 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4434 value = 3.6218 dict = 1.00 all = 3.7057 } Zip/UnZip{ index = 0.0958 value = 0.2761 dict = 1.00 all = 0.2699 } ---------------------------- total value len = 11.472340 GB avg = 0.155 KB (by entry num) total key len = 2.043874 GB avg = 0.028 KB total ukey len = 1.428630 GB avg = 0.016 KB total ukey num = 0.087088023 Billion total entry num = 0.073895752 Billion write speed all = 0.658544411 MB/sec (with seq num) write speed all = 0.629741295 MB/sec (without seq num) 2024/11/23-14:37:16.636670 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263786: 931083 keys, 36219488 bytes, temperature: kWarm 2024/11/23-14:37:16.636750 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343836636708, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263786, "file_size": 36219488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29860256, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343833, "oldest_key_time": 0, "file_creation_time": 1732343833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263786, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:16.678210 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst) /dev/shm/ToplingTemp/Topling-76419-NkbheE 2024/11/23-14:37:17.696558 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst): first pass time = 1.02's, 131.804'MB/sec 2024/11/23-14:37:18.974293 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst): old prealloc_size = 147639500, real_size = 32092992 2024/11/23-14:37:18.974872 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925740 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.98 avg-zval = 32.50 2024/11/23-14:37:18.975675 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:19.802306 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263787.sst): second pass time = 1.28's, 84.815'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.546'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.239'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 282.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30848.618'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.815'MB/sec zip pipeline throughput = 84.815'MB/sec entries = 925740 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.98 avg-zval = 32.50 usrkeys = 925740 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.16 avg-zval = 32.50 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 816382 bytes = 2959392 } seq expand size = 5714674 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4434 value = 3.5993 dict = 1.00 all = 3.7224 } Zip/UnZip{ index = 0.0958 value = 0.2778 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 11.586352 GB avg = 0.155 KB (by entry num) total key len = 2.069794 GB avg = 0.028 KB total ukey len = 1.447145 GB avg = 0.016 KB total ukey num = 0.088013763 Billion total entry num = 0.074821492 Billion write speed all = 0.665258211 MB/sec (with seq num) write speed all = 0.636098821 MB/sec (without seq num) 2024/11/23-14:37:19.847527 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263787: 925740 keys, 36056522 bytes, temperature: kWarm 2024/11/23-14:37:19.847600 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343839847564, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263787, "file_size": 36056522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30088496, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920720, "raw_average_key_size": 28, "raw_value_size": 108297148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343836, "oldest_key_time": 0, "file_creation_time": 1732343836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263787, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:19.898648 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst) /dev/shm/ToplingTemp/Topling-76419-oxudhC 2024/11/23-14:37:20.911218 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst): first pass time = 1.01's, 132.558'MB/sec 2024/11/23-14:37:22.473966 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst): old prealloc_size = 147639500, real_size = 36922208 2024/11/23-14:37:22.474560 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926242 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.91 avg-zval = 37.70 2024/11/23-14:37:22.474780 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:23.179426 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263788.sst): second pass time = 1.56's, 69.327'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.243'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.565'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.399'MB/sec write SST data time = 0.12's, 308.491'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35658.469'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.327'MB/sec zip pipeline throughput = 69.327'MB/sec entries = 926242 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 37.70 usrkeys = 926242 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.73 avg-zval = 37.70 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 108612 bytes = 339424 } seq expand size = 760284 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4450 value = 3.1012 dict = 1.00 all = 3.5076 } Zip/UnZip{ index = 0.0957 value = 0.3225 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 11.695395 GB avg = 0.154 KB (by entry num) total key len = 2.095729 GB avg = 0.028 KB total ukey len = 1.465669 GB avg = 0.016 KB total ukey num = 0.088940005 Billion total entry num = 0.075747734 Billion write speed all = 0.671723149 MB/sec (with seq num) write speed all = 0.642207640 MB/sec (without seq num) 2024/11/23-14:37:23.226368 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263788: 926242 keys, 38264729 bytes, temperature: kWarm 2024/11/23-14:37:23.226435 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343843226401, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263788, "file_size": 38264729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3271297785, "table_properties": {"data_size": 34916880, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934776, "raw_average_key_size": 28, "raw_value_size": 108283004, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343839, "oldest_key_time": 0, "file_creation_time": 1732343839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263788, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:23.266252 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst) /dev/shm/ToplingTemp/Topling-76419-FnmX7B 2024/11/23-14:37:24.311331 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst): first pass time = 1.05's, 128.434'MB/sec 2024/11/23-14:37:25.687347 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst): old prealloc_size = 147639500, real_size = 38037968 2024/11/23-14:37:25.687928 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925930 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0360 GB avg-val = 116.95 avg-zval = 38.92 2024/11/23-14:37:25.688499 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:26.431532 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263789.sst): second pass time = 1.37's, 78.769'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 55.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.579'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.374'MB/sec write SST data time = 0.12's, 330.691'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34740.324'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.769'MB/sec zip pipeline throughput = 78.769'MB/sec entries = 925930 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 38.92 usrkeys = 925930 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.12 avg-zval = 38.92 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 551325 bytes = 2067488 } seq expand size = 3859275 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0360 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4445 value = 3.0053 dict = 1.00 all = 3.2649 } Zip/UnZip{ index = 0.0957 value = 0.3327 dict = 1.00 all = 0.3063 } ---------------------------- total value len = 11.807546 GB avg = 0.154 KB (by entry num) total key len = 2.121655 GB avg = 0.028 KB total ukey len = 1.484188 GB avg = 0.017 KB total ukey num = 0.089865935 Billion total entry num = 0.076673664 Billion write speed all = 0.678341011 MB/sec (with seq num) write speed all = 0.648469441 MB/sec (without seq num) 2024/11/23-14:37:26.481445 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263789: 925930 keys, 41109006 bytes, temperature: kWarm 2024/11/23-14:37:26.481512 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343846481477, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263789, "file_size": 41109006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4028963475, "table_properties": {"data_size": 36033216, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926040, "raw_average_key_size": 28, "raw_value_size": 108291820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925930, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343843, "oldest_key_time": 0, "file_creation_time": 1732343843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263789, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:26.519853 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst) /dev/shm/ToplingTemp/Topling-76419-9HfDlC 2024/11/23-14:37:27.399569 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst): first pass time = 0.88's, 152.577'MB/sec 2024/11/23-14:37:28.736241 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst): old prealloc_size = 147639500, real_size = 38353728 2024/11/23-14:37:28.736822 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926580 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0364 GB avg-val = 116.85 avg-zval = 39.24 2024/11/23-14:37:28.737634 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:29.516700 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263790.sst): second pass time = 1.34's, 81.081'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 54.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.044'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.353'MB/sec write SST data time = 0.12's, 347.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35868.419'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.081'MB/sec zip pipeline throughput = 81.081'MB/sec entries = 926580 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.85 avg-zval = 39.24 usrkeys = 926580 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.85 avg-zval = 39.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926580 bytes = 3474688 } seq expand size = 6486060 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0364 GB dict = 1.08 MB all = 0.0427 GB } UnZip/Zip{ index = 10.5140 value = 2.9779 dict = 1.00 all = 3.1442 } Zip/UnZip{ index = 0.0951 value = 0.3358 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 11.922306 GB avg = 0.154 KB (by entry num) total key len = 2.147599 GB avg = 0.028 KB total ukey len = 1.502720 GB avg = 0.017 KB total ukey num = 0.090792515 Billion total entry num = 0.077600244 Billion write speed all = 0.685090249 MB/sec (with seq num) write speed all = 0.654862230 MB/sec (without seq num) 2024/11/23-14:37:29.567602 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263790: 926580 keys, 42686988 bytes, temperature: kWarm 2024/11/23-14:37:29.567679 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343849567636, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263790, "file_size": 42686988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236084454, "largest_seqno": 4028963334, "table_properties": {"data_size": 36359344, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944240, "raw_average_key_size": 28, "raw_value_size": 108273616, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926580, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343846, "oldest_key_time": 0, "file_creation_time": 1732343846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263790, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:29.607683 139957383018048 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst) /dev/shm/ToplingTemp/Topling-76419-lAtqWF 2024/11/23-14:37:29.724705 139957383018048 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst): first pass time = 0.12's, 241.488'MB/sec 2024/11/23-14:37:29.985220 139957383018048 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst): old prealloc_size = 147639500, real_size = 8377168 2024/11/23-14:37:29.985396 139957383018048 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 195907 prefix = 4 raw-key = 0.0031 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0228 GB zip-val = 0.0080 GB avg-val = 116.21 avg-zval = 40.60 2024/11/23-14:37:29.985593 139957383018048 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/23-14:37:30.149969 139957383018048 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00/263791.sst): second pass time = 0.26's, 87.626'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 55.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2408.569'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.356'MB/sec write SST data time = 0.03's, 337.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.237'MB, throughput = 8583.638'MB/sec zip my value time = 0.26's, unzip length = 0.023'GB zip my value throughput = 87.626'MB/sec zip pipeline throughput = 87.626'MB/sec entries = 195907 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 40.60 usrkeys = 195907 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.21 avg-zval = 40.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 195907 bytes = 538752 } seq expand size = 1371349 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0039 GB value = 0.0228 GB dict = 0.24 MB all = 0.0283 GB } __ZipSize{ index = 0.0004 GB value = 0.0080 GB dict = 0.24 MB all = 0.0091 GB } UnZip/Zip{ index = 10.4777 value = 2.8622 dict = 1.00 all = 3.1024 } Zip/UnZip{ index = 0.0954 value = 0.3494 dict = 1.00 all = 0.3223 } ---------------------------- total value len = 11.946443 GB avg = 0.154 KB (by entry num) total key len = 2.153085 GB avg = 0.028 KB total ukey len = 1.506638 GB avg = 0.017 KB total ukey num = 0.090988422 Billion total entry num = 0.077796151 Billion write speed all = 0.686511456 MB/sec (with seq num) write speed all = 0.656208059 MB/sec (without seq num) 2024/11/23-14:37:30.166846 139957383018048 [compaction_job.cc:1948] [default] [JOB 28041] Generated table #263791: 195907 keys, 9106361 bytes, temperature: kWarm 2024/11/23-14:37:30.166904 139957383018048 EVENT_LOG_v1 {"time_micros": 1732343850166873, "cf_name": "default", "job": 28041, "event": "table_file_creation", "file_number": 263791, "file_size": 9106361, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4025935883, "largest_seqno": 4028963445, "table_properties": {"data_size": 7954048, "index_size": 373952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5485396, "raw_average_key_size": 28, "raw_value_size": 22765774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 195907, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343849, "oldest_key_time": 0, "file_creation_time": 1732343849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263791, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:30.179516 139957383018048 [compaction_job.cc:703] job-28041: subcompact[0], size: 1.028393 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28041/att-00 [263764,263765,263766,263767,263768,263769,263770,263771,263772,263773,263774,263775,263776,263777,263778,263779,263780,263781,263782,263783,263784,263785,263786,263787,263788,263789,263790,263791] 2024/11/23-14:37:30.343219 139957383018048 (Original Log Time 2024/11/23-14:36:02.633519) EVENT_LOG_v1 {"time_micros": 1732343762633479, "job": 28041, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [263706], "files_L3": [263727, 263728, 263657, 263658, 263659, 263660, 263661, 263662, 263663, 263664, 263665, 263666, 263667, 263668, 263669, 263670, 263671, 263672, 263673, 263674, 263675, 263676, 263677, 263678, 263679, 263680, 263681, 263760], "score": 0.995847, "input_data_size": 1050429466, "oldest_snapshot_seqno": 4029595512} 2024/11/23-14:37:30.343223 139957383018048 (Original Log Time 2024/11/23-14:37:30.341636) [compaction_job.cc:2022] [default] [JOB 28041] Compacted 1@2 + 28@3 files to L3 => 1028393202 bytes 2024/11/23-14:37:30.343224 139957383018048 (Original Log Time 2024/11/23-14:37:30.343146) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 28 0 0 0] max score 0.04, MB/sec: 12.0 rd, 11.7 wr, level 3, files in(1, 28) out(28 +0 blob) MB in(27.2, 974.6 +0.0 blob) out(980.8 +0.0 blob), read-write-amplify(73.0) write-amplify(36.1) OK, records in: 25538481, records dropped: 100494 output_compression: Snappy 2024/11/23-14:37:30.343228 139957383018048 (Original Log Time 2024/11/23-14:37:30.343193) EVENT_LOG_v1 {"time_micros": 1732343850343170, "cf": "default", "job": 28041, "event": "compaction_finished", "compaction_time_micros": 87546002, "compaction_time_cpu_micros": 37511221, "output_level": 3, "num_output_files": 28, "total_output_size": 1028393202, "num_input_records": 25538481, "num_output_records": 25437987, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 703883013, "file_range_sync_nanos": 0, "file_fsync_nanos": 20683, "file_prepare_write_nanos": 79960, "lsm_state": [0, 0, 0, 28, 0, 0, 0]} 2024/11/23-14:37:30.455326 139957383018048 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732343762,"executesMs":87856,"compactionJobId":28041,"attempt":0,"compactionInputRawBytes":3666402199,"compactionInputZipBytes":1050429466,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1028393202,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-10b61c6d3eb2fa308d9fafd4cb7951fd-0eb5c4263b282a83-00"}}, response = 2024/11/23-14:37:30.455400 139957383018048 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28041/att-00: olev 3, work 87.856 s, result 7.720 ms, install 1.605 ms, input{raw 3.415 GiB zip 1001.768 MiB}