2024/11/23-13:23:56.981700 139957584442944 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00: fork to child time = 0.021269 sec 2024/11/23-13:23:56.982683 139957584442944 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00 2024/11/23-13:23:56.990708 139957584442944 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00 2024/11/23-13:23:56.991934 139957584442944 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-13:23:56.992299 139957584442944 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-13:23:57.044848 139957584442944 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26755/att-00/MANIFEST-206347 2024/11/23-13:23:57.078355 139957584442944 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26755/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 261089, last_sequence is 3994688239, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 260931 2024/11/23-13:23:57.078367 139957584442944 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-13:23:57.078487 139957584442944 [version_set.cc:5721] Creating manifest 261089 2024/11/23-13:23:57.116373 139957584442944 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-13:23:57.138242 139957584442944 [compaction_job.cc:2437] [default] [JOB 26755] Compacting 1@2 + 15@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-13:23:57.138256 139957584442944 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20042 Base level 2, inputs: [261074(27MB)], [261037(35MB) 261038(33MB) 261039(32MB) 261040(33MB) 261041(36MB) 261042(33MB) 261043(36MB) 261044(35MB) 261045(10MB) 261012(34MB) 261013(34MB) 261014(32MB) 261015(34MB) 261016(34MB) 261086(33MB)] 2024/11/23-13:23:57.144954 139957584442944 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:23:57.144992 139957584442944 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:23:57.145012 139957584442944 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:23:57.145018 139957584442944 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:23:57.145028 139957584442944 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:23:58.002769 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst) /dev/shm/ToplingTemp/Topling-59200-XOIfJD 2024/11/23-13:23:58.564972 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst): first pass time = 0.56's, 238.758'MB/sec 2024/11/23-13:24:00.343877 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst): old prealloc_size = 147639500, real_size = 32686224 2024/11/23-13:24:00.344434 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930945 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.17 avg-zval = 32.93 2024/11/23-13:24:00.345326 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:01.321643 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261090.sst): second pass time = 1.78's, 60.830'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 37.872'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.632'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.276'MB/sec write SST data time = 0.13's, 279.552'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 41841.123'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.830'MB/sec zip pipeline throughput = 113.403'MB/sec entries = 930945 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.17 avg-zval = 32.93 usrkeys = 930945 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.17 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930945 bytes = 3491056 } seq expand size = 6516615 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3744 value = 3.5276 dict = 1.00 all = 3.6247 } Zip/UnZip{ index = 0.0964 value = 0.2835 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 8.963288 GB avg = 0.168 KB (by entry num) total key len = 1.466888 GB avg = 0.028 KB total ukey len = 1.016497 GB avg = 0.015 KB total ukey num = 0.066481380 Billion total entry num = 0.053289109 Billion write speed all = 0.646667874 MB/sec (with seq num) write speed all = 0.620236599 MB/sec (without seq num) 2024/11/23-13:24:01.366919 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261090: 930945 keys, 37029122 bytes, temperature: kWarm 2024/11/23-13:24:01.367035 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339441366955, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261090, "file_size": 37029122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3994639976, "table_properties": {"data_size": 30658624, "index_size": 1794688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066460, "raw_average_key_size": 28, "raw_value_size": 108151390, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930945, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339437, "oldest_key_time": 0, "file_creation_time": 1732339437, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261090, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:01.405058 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst) /dev/shm/ToplingTemp/Topling-59200-oEsd9C 2024/11/23-13:24:02.289826 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst): first pass time = 0.88's, 151.704'MB/sec 2024/11/23-13:24:03.851677 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst): old prealloc_size = 147639500, real_size = 30680608 2024/11/23-13:24:03.852314 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931224 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 116.13 avg-zval = 30.78 2024/11/23-13:24:03.853188 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:04.830304 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261091.sst): second pass time = 1.56's, 69.276'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 35.670'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2900.980'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.14's, 240.949'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37223.170'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.276'MB/sec zip pipeline throughput = 69.276'MB/sec entries = 931224 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.13 avg-zval = 30.78 usrkeys = 931224 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.13 avg-zval = 30.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931224 bytes = 3375696 } seq expand size = 6518568 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4435 value = 3.7728 dict = 1.00 all = 3.8449 } Zip/UnZip{ index = 0.0958 value = 0.2651 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 9.077950 GB avg = 0.167 KB (by entry num) total key len = 1.492962 GB avg = 0.028 KB total ukey len = 1.035121 GB avg = 0.015 KB total ukey num = 0.067412604 Billion total entry num = 0.054220333 Billion write speed all = 0.655250950 MB/sec (with seq num) write speed all = 0.628363639 MB/sec (without seq num) 2024/11/23-13:24:04.873413 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261091: 931224 keys, 34907646 bytes, temperature: kWarm 2024/11/23-13:24:04.873481 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339444873448, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261091, "file_size": 34907646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3994641359, "table_properties": {"data_size": 28664272, "index_size": 1783360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074272, "raw_average_key_size": 28, "raw_value_size": 108143568, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931224, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339441, "oldest_key_time": 0, "file_creation_time": 1732339441, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261091, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:04.918072 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst) /dev/shm/ToplingTemp/Topling-59200-QWagUB 2024/11/23-13:24:06.018497 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst): first pass time = 1.10's, 121.974'MB/sec 2024/11/23-13:24:07.787264 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst): old prealloc_size = 147639500, real_size = 30318832 2024/11/23-13:24:07.787845 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929938 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.33 avg-zval = 30.44 2024/11/23-13:24:07.788620 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:08.737471 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261092.sst): second pass time = 1.77's, 61.198'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 36.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.587'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 273.580'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36974.340'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.198'MB/sec zip pipeline throughput = 111.772'MB/sec entries = 929938 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.33 avg-zval = 30.44 usrkeys = 929938 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.33 avg-zval = 30.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929938 bytes = 3138560 } seq expand size = 6509566 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.4452 value = 3.8218 dict = 1.00 all = 3.9121 } Zip/UnZip{ index = 0.0957 value = 0.2617 dict = 1.00 all = 0.2556 } ---------------------------- total value len = 9.192639 GB avg = 0.167 KB (by entry num) total key len = 1.519000 GB avg = 0.028 KB total ukey len = 1.053720 GB avg = 0.015 KB total ukey num = 0.068342542 Billion total entry num = 0.055150271 Billion write speed all = 0.663813340 MB/sec (with seq num) write speed all = 0.636471504 MB/sec (without seq num) 2024/11/23-13:24:08.779434 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261092: 929938 keys, 34308188 bytes, temperature: kWarm 2024/11/23-13:24:08.779508 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339448779469, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261092, "file_size": 34308188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3888868977, "largest_seqno": 3994641211, "table_properties": {"data_size": 28305568, "index_size": 1780608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038264, "raw_average_key_size": 28, "raw_value_size": 108179536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929938, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339444, "oldest_key_time": 0, "file_creation_time": 1732339444, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261092, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:08.816824 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst) /dev/shm/ToplingTemp/Topling-59200-VP28uD 2024/11/23-13:24:09.808268 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst): first pass time = 0.99's, 135.384'MB/sec 2024/11/23-13:24:11.627234 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst): old prealloc_size = 147639500, real_size = 30827328 2024/11/23-13:24:11.627816 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930178 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0288 GB avg-val = 116.29 avg-zval = 30.98 2024/11/23-13:24:11.628639 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:12.599962 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261093.sst): second pass time = 1.82's, 59.497'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 36.035'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3173.218'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.275'MB/sec write SST data time = 0.13's, 276.302'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34839.828'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.497'MB/sec zip pipeline throughput = 131.895'MB/sec entries = 930178 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.29 avg-zval = 30.98 usrkeys = 930178 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.29 avg-zval = 30.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930178 bytes = 3371904 } seq expand size = 6511246 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4441 value = 3.7543 dict = 1.00 all = 3.8291 } Zip/UnZip{ index = 0.0957 value = 0.2664 dict = 1.00 all = 0.2612 } ---------------------------- total value len = 9.307323 GB avg = 0.166 KB (by entry num) total key len = 1.545045 GB avg = 0.028 KB total ukey len = 1.072324 GB avg = 0.015 KB total ukey num = 0.069272720 Billion total entry num = 0.056080449 Billion write speed all = 0.672373547 MB/sec (with seq num) write speed all = 0.644577210 MB/sec (without seq num) 2024/11/23-13:24:12.645928 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261093: 930178 keys, 35051654 bytes, temperature: kWarm 2024/11/23-13:24:12.646009 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339452645964, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261093, "file_size": 35051654, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3994641223, "table_properties": {"data_size": 28813360, "index_size": 1781248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044984, "raw_average_key_size": 28, "raw_value_size": 108172784, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339448, "oldest_key_time": 0, "file_creation_time": 1732339448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261093, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:12.683028 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst) /dev/shm/ToplingTemp/Topling-59200-9NUDYC 2024/11/23-13:24:13.734374 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst): first pass time = 1.05's, 127.667'MB/sec 2024/11/23-13:24:15.150634 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst): old prealloc_size = 147639500, real_size = 33961024 2024/11/23-13:24:15.151351 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926808 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 116.82 avg-zval = 34.49 2024/11/23-13:24:15.152273 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:16.812836 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261094.sst): second pass time = 1.42's, 76.495'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.66's, 23.037'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2534.142'MB/sec (index lex order gen) rebuild zvType time = 1.40's, 0.165'MB/sec write SST data time = 0.26's, 146.654'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35256.013'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.495'MB/sec zip pipeline throughput = 76.495'MB/sec entries = 926808 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 34.49 usrkeys = 926808 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 34.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926808 bytes = 3475552 } seq expand size = 6487656 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5139 value = 3.3869 dict = 1.00 all = 3.5052 } Zip/UnZip{ index = 0.0951 value = 0.2953 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 9.422078 GB avg = 0.165 KB (by entry num) total key len = 1.570996 GB avg = 0.028 KB total ukey len = 1.090860 GB avg = 0.016 KB total ukey num = 0.070199528 Billion total entry num = 0.057007257 Billion write speed all = 0.680913426 MB/sec (with seq num) write speed all = 0.652665089 MB/sec (without seq num) 2024/11/23-13:24:16.861924 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261094: 926808 keys, 38291054 bytes, temperature: kWarm 2024/11/23-13:24:16.862023 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339456861965, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261094, "file_size": 38291054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3994641429, "table_properties": {"data_size": 31966128, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950624, "raw_average_key_size": 28, "raw_value_size": 108267244, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926808, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339452, "oldest_key_time": 0, "file_creation_time": 1732339452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261094, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:16.930946 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst) /dev/shm/ToplingTemp/Topling-59200-LSmoZA 2024/11/23-13:24:18.005557 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst): first pass time = 1.07's, 124.906'MB/sec 2024/11/23-13:24:19.365927 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst): old prealloc_size = 147639500, real_size = 31302048 2024/11/23-13:24:19.366583 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932346 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 115.96 avg-zval = 31.42 2024/11/23-13:24:19.367342 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:20.754968 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261095.sst): second pass time = 1.36's, 79.546'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.39's, 25.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2813.640'MB/sec (index lex order gen) rebuild zvType time = 1.13's, 0.207'MB/sec write SST data time = 0.26's, 134.510'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30784.625'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.546'MB/sec zip pipeline throughput = 79.546'MB/sec entries = 932346 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.96 avg-zval = 31.42 usrkeys = 932346 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.06 avg-zval = 31.42 TagRS{ kinds = RS_Key0_TagN bytes = 145760 } TagArray{ size = 812628 bytes = 2742640 } seq expand size = 5688396 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5130 value = 3.6905 dict = 1.00 all = 3.8299 } Zip/UnZip{ index = 0.0951 value = 0.2710 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 9.535878 GB avg = 0.165 KB (by entry num) total key len = 1.597101 GB avg = 0.028 KB total ukey len = 1.109507 GB avg = 0.016 KB total ukey num = 0.071131874 Billion total entry num = 0.057939603 Billion write speed all = 0.689410907 MB/sec (with seq num) write speed all = 0.660707580 MB/sec (without seq num) 2024/11/23-13:24:20.800552 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261095: 932346 keys, 35044944 bytes, temperature: kWarm 2024/11/23-13:24:20.800653 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339460800604, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261095, "file_size": 35044944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3994641315, "table_properties": {"data_size": 29295056, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105688, "raw_average_key_size": 28, "raw_value_size": 108112080, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932346, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339456, "oldest_key_time": 0, "file_creation_time": 1732339456, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261095, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:20.852502 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst) /dev/shm/ToplingTemp/Topling-59200-EbeXyD 2024/11/23-13:24:21.948630 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst): first pass time = 1.10's, 122.453'MB/sec 2024/11/23-13:24:23.300110 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst): old prealloc_size = 147639500, real_size = 33914080 2024/11/23-13:24:23.300692 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926660 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.84 avg-zval = 34.46 2024/11/23-13:24:23.301487 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:24.484807 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261096.sst): second pass time = 1.35's, 80.159'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.18's, 32.208'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.382'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.28's, 136.641'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34797.874'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.159'MB/sec zip pipeline throughput = 80.159'MB/sec entries = 926660 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.84 avg-zval = 34.46 usrkeys = 926660 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.36 avg-zval = 34.46 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 863381 bytes = 3237696 } seq expand size = 6043667 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5841 value = 3.3908 dict = 1.00 all = 3.5177 } Zip/UnZip{ index = 0.0945 value = 0.2949 dict = 1.00 all = 0.2843 } ---------------------------- total value len = 9.650193 GB avg = 0.164 KB (by entry num) total key len = 1.623048 GB avg = 0.028 KB total ukey len = 1.128040 GB avg = 0.016 KB total ukey num = 0.072058534 Billion total entry num = 0.058866263 Billion write speed all = 0.697935407 MB/sec (with seq num) write speed all = 0.668779747 MB/sec (without seq num) 2024/11/23-13:24:24.534757 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261096: 926660 keys, 38154986 bytes, temperature: kWarm 2024/11/23-13:24:24.534851 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339464534803, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261096, "file_size": 38154986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3994641940, "table_properties": {"data_size": 31931216, "index_size": 1751040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946480, "raw_average_key_size": 28, "raw_value_size": 108271272, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926660, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339460, "oldest_key_time": 0, "file_creation_time": 1732339460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261096, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:24.613475 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst) /dev/shm/ToplingTemp/Topling-59200-blN8kC 2024/11/23-13:24:25.778206 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst): first pass time = 1.16's, 115.240'MB/sec 2024/11/23-13:24:27.352480 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst): old prealloc_size = 147639500, real_size = 33010352 2024/11/23-13:24:27.353211 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936279 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0310 GB avg-val = 115.35 avg-zval = 33.09 2024/11/23-13:24:27.354103 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:28.251454 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261097.sst): second pass time = 1.57's, 68.669'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.440'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2536.407'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.307'MB/sec write SST data time = 0.14's, 272.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36952.296'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.669'MB/sec zip pipeline throughput = 68.669'MB/sec entries = 936279 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.35 avg-zval = 33.09 usrkeys = 936279 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.35 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936279 bytes = 3394032 } seq expand size = 6553953 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4451 value = 3.4858 dict = 1.00 all = 3.6029 } Zip/UnZip{ index = 0.0957 value = 0.2869 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 9.764749 GB avg = 0.163 KB (by entry num) total key len = 1.649264 GB avg = 0.028 KB total ukey len = 1.146766 GB avg = 0.016 KB total ukey num = 0.072994813 Billion total entry num = 0.059802542 Billion write speed all = 0.706485947 MB/sec (with seq num) write speed all = 0.676873466 MB/sec (without seq num) 2024/11/23-13:24:28.298949 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261097: 936279 keys, 37252924 bytes, temperature: kWarm 2024/11/23-13:24:28.299024 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339468298985, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261097, "file_size": 37252924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3994642315, "table_properties": {"data_size": 30983328, "index_size": 1792768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26215812, "raw_average_key_size": 28, "raw_value_size": 108001974, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936279, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339464, "oldest_key_time": 0, "file_creation_time": 1732339464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261097, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:28.346330 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst) /dev/shm/ToplingTemp/Topling-59200-QMhTjB 2024/11/23-13:24:29.434027 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst): first pass time = 1.09's, 123.400'MB/sec 2024/11/23-13:24:31.390835 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst): old prealloc_size = 147639500, real_size = 31904112 2024/11/23-13:24:31.391397 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927105 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.77 avg-zval = 32.26 2024/11/23-13:24:31.392204 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:32.509127 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261098.sst): second pass time = 1.96's, 55.345'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 32.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.335'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.23's, 155.682'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29767.571'MB/sec zip my value time = 1.96's, unzip length = 0.108'GB zip my value throughput = 55.345'MB/sec zip pipeline throughput = 104.274'MB/sec entries = 927105 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 32.26 usrkeys = 927105 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 32.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927105 bytes = 3360768 } seq expand size = 6489735 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5127 value = 3.6197 dict = 1.00 all = 3.7159 } Zip/UnZip{ index = 0.0951 value = 0.2763 dict = 1.00 all = 0.2691 } ---------------------------- total value len = 9.879498 GB avg = 0.163 KB (by entry num) total key len = 1.675223 GB avg = 0.028 KB total ukey len = 1.165308 GB avg = 0.016 KB total ukey num = 0.073921918 Billion total entry num = 0.060729647 Billion write speed all = 0.715006806 MB/sec (with seq num) write speed all = 0.684943172 MB/sec (without seq num) 2024/11/23-13:24:32.558649 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261098: 927105 keys, 36119406 bytes, temperature: kWarm 2024/11/23-13:24:32.558735 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339472558686, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261098, "file_size": 36119406, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3994641884, "table_properties": {"data_size": 29908384, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958940, "raw_average_key_size": 28, "raw_value_size": 108258882, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927105, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339468, "oldest_key_time": 0, "file_creation_time": 1732339468, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261098, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:32.625258 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst) /dev/shm/ToplingTemp/Topling-59200-zkJLJB 2024/11/23-13:24:33.717941 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst): first pass time = 1.09's, 122.838'MB/sec 2024/11/23-13:24:35.618409 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst): old prealloc_size = 147639500, real_size = 32410544 2024/11/23-13:24:35.618945 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930694 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.21 avg-zval = 32.65 2024/11/23-13:24:35.619778 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:36.701449 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261099.sst): second pass time = 1.90's, 56.932'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3472.394'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.244'MB/sec write SST data time = 0.13's, 284.562'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36244.529'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.932'MB/sec zip pipeline throughput = 80.951'MB/sec entries = 930694 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.21 avg-zval = 32.65 usrkeys = 930694 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.21 avg-zval = 32.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930694 bytes = 3490112 } seq expand size = 6514858 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3765 value = 3.5597 dict = 1.00 all = 3.6520 } Zip/UnZip{ index = 0.0964 value = 0.2809 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 9.994171 GB avg = 0.162 KB (by entry num) total key len = 1.701282 GB avg = 0.028 KB total ukey len = 1.183922 GB avg = 0.016 KB total ukey num = 0.074852612 Billion total entry num = 0.061660341 Billion write speed all = 0.723527654 MB/sec (with seq num) write speed all = 0.693011205 MB/sec (without seq num) 2024/11/23-13:24:36.748458 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261099: 930694 keys, 36751974 bytes, temperature: kWarm 2024/11/23-13:24:36.748526 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339476748491, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261099, "file_size": 36751974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3994641991, "table_properties": {"data_size": 30383840, "index_size": 1793856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059432, "raw_average_key_size": 28, "raw_value_size": 108158388, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930694, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339472, "oldest_key_time": 0, "file_creation_time": 1732339472, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261099, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:36.787152 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst) /dev/shm/ToplingTemp/Topling-59200-IDSLtE 2024/11/23-13:24:37.698143 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst): first pass time = 0.91's, 147.338'MB/sec 2024/11/23-13:24:39.357695 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst): old prealloc_size = 147639500, real_size = 32958992 2024/11/23-13:24:39.358288 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924711 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.15 avg-zval = 33.49 2024/11/23-13:24:39.359131 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:40.403599 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261100.sst): second pass time = 1.66's, 65.307'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3023.535'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.14's, 263.872'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32613.221'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.307'MB/sec zip pipeline throughput = 65.307'MB/sec entries = 924711 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 33.49 usrkeys = 924711 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924711 bytes = 3352096 } seq expand size = 6472977 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5104 value = 3.4980 dict = 1.00 all = 3.6112 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 10.108970 GB avg = 0.162 KB (by entry num) total key len = 1.727174 GB avg = 0.028 KB total ukey len = 1.202416 GB avg = 0.016 KB total ukey num = 0.075777323 Billion total entry num = 0.062585052 Billion write speed all = 0.732063684 MB/sec (with seq num) write speed all = 0.701096677 MB/sec (without seq num) 2024/11/23-13:24:40.455020 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261100: 924711 keys, 37166612 bytes, temperature: kWarm 2024/11/23-13:24:40.455106 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339480455055, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261100, "file_size": 37166612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3994641337, "table_properties": {"data_size": 30968000, "index_size": 1759616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891908, "raw_average_key_size": 28, "raw_value_size": 108325870, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924711, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339476, "oldest_key_time": 0, "file_creation_time": 1732339476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261100, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:40.494584 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst) /dev/shm/ToplingTemp/Topling-59200-IRnY3A 2024/11/23-13:24:41.589057 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst): first pass time = 1.09's, 122.636'MB/sec 2024/11/23-13:24:43.068217 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst): old prealloc_size = 147639500, real_size = 32908864 2024/11/23-13:24:43.068793 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931197 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.13 avg-zval = 33.19 2024/11/23-13:24:43.069595 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:44.184923 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261101.sst): second pass time = 1.48's, 73.147'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.255'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.568'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.240'MB/sec write SST data time = 0.15's, 251.069'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35273.483'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.147'MB/sec zip pipeline throughput = 73.147'MB/sec entries = 931197 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 33.19 usrkeys = 931197 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931197 bytes = 3375600 } seq expand size = 6518379 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5145 value = 3.4993 dict = 1.00 all = 3.6143 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 10.223633 GB avg = 0.161 KB (by entry num) total key len = 1.753247 GB avg = 0.028 KB total ukey len = 1.221040 GB avg = 0.016 KB total ukey num = 0.076708520 Billion total entry num = 0.063516249 Billion write speed all = 0.740594989 MB/sec (with seq num) write speed all = 0.709174576 MB/sec (without seq num) 2024/11/23-13:24:44.233483 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261101: 931197 keys, 37135622 bytes, temperature: kWarm 2024/11/23-13:24:44.233581 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339484233522, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261101, "file_size": 37135622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3596991077, "largest_seqno": 3994640729, "table_properties": {"data_size": 30904624, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26073516, "raw_average_key_size": 28, "raw_value_size": 108144290, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931197, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339480, "oldest_key_time": 0, "file_creation_time": 1732339480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261101, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:44.271437 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst) /dev/shm/ToplingTemp/Topling-59200-XNIFbB 2024/11/23-13:24:45.507649 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst): first pass time = 1.24's, 108.575'MB/sec 2024/11/23-13:24:46.937169 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst): old prealloc_size = 147639500, real_size = 32612864 2024/11/23-13:24:46.937871 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931335 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.11 avg-zval = 32.86 2024/11/23-13:24:46.938767 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:47.897189 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261102.sst): second pass time = 1.43's, 75.694'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2608.935'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.289'MB/sec write SST data time = 0.15's, 241.524'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35238.848'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.694'MB/sec zip pipeline throughput = 75.694'MB/sec entries = 931335 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 32.86 usrkeys = 931335 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 32.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931335 bytes = 3492528 } seq expand size = 6519345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5119 value = 3.5331 dict = 1.00 all = 3.6318 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 10.338292 GB avg = 0.160 KB (by entry num) total key len = 1.779325 GB avg = 0.028 KB total ukey len = 1.239666 GB avg = 0.016 KB total ukey num = 0.077639855 Billion total entry num = 0.064447584 Billion write speed all = 0.749125562 MB/sec (with seq num) write speed all = 0.717251749 MB/sec (without seq num) 2024/11/23-13:24:47.944586 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261102: 931335 keys, 36956696 bytes, temperature: kWarm 2024/11/23-13:24:47.944654 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339487944620, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261102, "file_size": 36956696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3994642363, "table_properties": {"data_size": 30607856, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077380, "raw_average_key_size": 28, "raw_value_size": 108140398, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339484, "oldest_key_time": 0, "file_creation_time": 1732339484, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261102, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:47.980438 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst) /dev/shm/ToplingTemp/Topling-59200-hilQuD 2024/11/23-13:24:48.806027 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst): first pass time = 0.83's, 162.580'MB/sec 2024/11/23-13:24:50.125732 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst): old prealloc_size = 147639500, real_size = 33125584 2024/11/23-13:24:50.126303 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929761 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.36 avg-zval = 33.46 2024/11/23-13:24:50.127089 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:24:50.982896 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261103.sst): second pass time = 1.32's, 82.026'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.995'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 296.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35226.476'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.026'MB/sec zip pipeline throughput = 82.026'MB/sec entries = 929761 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.36 avg-zval = 33.46 usrkeys = 929761 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.36 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929761 bytes = 3370400 } seq expand size = 6508327 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4439 value = 3.4772 dict = 1.00 all = 3.5936 } Zip/UnZip{ index = 0.0957 value = 0.2876 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 10.452985 GB avg = 0.160 KB (by entry num) total key len = 1.805358 GB avg = 0.028 KB total ukey len = 1.258262 GB avg = 0.016 KB total ukey num = 0.078569616 Billion total entry num = 0.065377345 Billion write speed all = 0.757680881 MB/sec (with seq num) write speed all = 0.725353404 MB/sec (without seq num) 2024/11/23-13:24:51.031038 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261103: 929761 keys, 37348696 bytes, temperature: kWarm 2024/11/23-13:24:51.031105 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339491031071, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261103, "file_size": 37348696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3994642297, "table_properties": {"data_size": 31112448, "index_size": 1780480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033308, "raw_average_key_size": 28, "raw_value_size": 108184466, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929761, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339487, "oldest_key_time": 0, "file_creation_time": 1732339487, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261103, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:51.075281 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst) /dev/shm/ToplingTemp/Topling-59200-Rzc6oE 2024/11/23-13:24:51.305295 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst): first pass time = 0.23's, 243.667'MB/sec 2024/11/23-13:24:51.863066 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst): old prealloc_size = 147639500, real_size = 14601024 2024/11/23-13:24:51.863341 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 386687 prefix = 4 raw-key = 0.0062 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0452 GB zip-val = 0.0138 GB avg-val = 116.91 avg-zval = 35.58 2024/11/23-13:24:51.863813 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.056 GB 2024/11/23-13:24:52.219132 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00/261104.sst): second pass time = 0.56's, 81.168'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.36's, 45.835'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2975.599'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.320'MB/sec write SST data time = 0.05's, 305.502'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.460'MB, throughput = 14362.642'MB/sec zip my value time = 0.56's, unzip length = 0.045'GB zip my value throughput = 81.168'MB/sec zip pipeline throughput = 81.168'MB/sec entries = 386687 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.91 avg-zval = 35.58 usrkeys = 386687 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.91 avg-zval = 35.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 386687 bytes = 1353424 } seq expand size = 2706809 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0077 GB value = 0.0452 GB dict = 0.46 MB all = 0.0560 GB } __ZipSize{ index = 0.0007 GB value = 0.0138 GB dict = 0.46 MB all = 0.0163 GB } UnZip/Zip{ index = 10.3645 value = 3.2861 dict = 1.00 all = 3.4337 } Zip/UnZip{ index = 0.0965 value = 0.3043 dict = 1.00 all = 0.2912 } ---------------------------- total value len = 10.500902 GB avg = 0.160 KB (by entry num) total key len = 1.816185 GB avg = 0.028 KB total ukey len = 1.265995 GB avg = 0.016 KB total ukey num = 0.078956303 Billion total entry num = 0.065764032 Billion write speed all = 0.761253615 MB/sec (with seq num) write speed all = 0.728737415 MB/sec (without seq num) 2024/11/23-13:24:52.244269 139957584442944 [compaction_job.cc:1948] [default] [JOB 26755] Generated table #261104: 386687 keys, 16319679 bytes, temperature: kWarm 2024/11/23-13:24:52.244335 139957584442944 EVENT_LOG_v1 {"time_micros": 1732339492244304, "cf_name": "default", "job": 26755, "event": "table_file_creation", "file_number": 261104, "file_size": 16319679, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3791294747, "largest_seqno": 3994642224, "table_properties": {"data_size": 13758000, "index_size": 746176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10827236, "raw_average_key_size": 28, "raw_value_size": 45209486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 386687, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732339491, "oldest_key_time": 0, "file_creation_time": 1732339491, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 261104, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:24:52.261156 139957584442944 [compaction_job.cc:703] job-26755: subcompact[0], size: 0.527839 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26755/att-00 [261090,261091,261092,261093,261094,261095,261096,261097,261098,261099,261100,261101,261102,261103,261104] 2024/11/23-13:24:52.373053 139957584442944 (Original Log Time 2024/11/23-13:23:57.138325) EVENT_LOG_v1 {"time_micros": 1732339437138270, "job": 26755, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [261074], "files_L3": [261037, 261038, 261039, 261040, 261041, 261042, 261043, 261044, 261045, 261012, 261013, 261014, 261015, 261016, 261086], "score": 0.987258, "input_data_size": 544290487, "oldest_snapshot_seqno": 3994688190} 2024/11/23-13:24:52.373057 139957584442944 (Original Log Time 2024/11/23-13:24:52.371844) [compaction_job.cc:2022] [default] [JOB 26755] Compacted 1@2 + 15@3 files to L3 => 527839203 bytes 2024/11/23-13:24:52.373058 139957584442944 (Original Log Time 2024/11/23-13:24:52.372972) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 9.9 rd, 9.6 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 492.0 +0.0 blob) out(503.4 +0.0 blob), read-write-amplify(37.7) write-amplify(18.6) OK, records in: 13458524, records dropped: 52656 output_compression: Snappy 2024/11/23-13:24:52.373064 139957584442944 (Original Log Time 2024/11/23-13:24:52.373032) EVENT_LOG_v1 {"time_micros": 1732339492373011, "cf": "default", "job": 26755, "event": "compaction_finished", "compaction_time_micros": 55122842, "compaction_time_cpu_micros": 23940849, "output_level": 3, "num_output_files": 15, "total_output_size": 527839203, "num_input_records": 13458524, "num_output_records": 13405868, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 426170750, "file_range_sync_nanos": 0, "file_fsync_nanos": 13483, "file_prepare_write_nanos": 43814, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-13:24:52.504705 139957584442944 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732339436,"executesMs":55382,"compactionJobId":26755,"attempt":0,"compactionInputRawBytes":1942563084,"compactionInputZipBytes":544290487,"compactionOutputRawBytes":0,"compactionOutputZipBytes":527839203,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0a1ee2ce79c65d2d9c391dc3c93f72ce-fe02a1aae920338f-00"}}, response = 2024/11/23-13:24:52.504804 139957584442944 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26755/att-00: olev 3, work 55.382 s, result 7.637 ms, install 1.236 ms, input{raw 1.809 GiB zip 519.076 MiB}