2024/11/23-17:07:43.550632 139957215163968 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00: fork to child time = 0.007979 sec 2024/11/23-17:07:43.551369 139957215163968 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00 2024/11/23-17:07:43.558995 139957215163968 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00 2024/11/23-17:07:43.559881 139957215163968 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:07:43.560148 139957215163968 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:07:43.615083 139957215163968 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31522/att-00/MANIFEST-206347 2024/11/23-17:07:43.650293 139957215163968 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31522/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 271385, last_sequence is 4134984150, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271215 2024/11/23-17:07:43.650306 139957215163968 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:07:43.650434 139957215163968 [version_set.cc:5721] Creating manifest 271385 2024/11/23-17:07:43.685668 139957215163968 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:07:43.709913 139957215163968 [compaction_job.cc:2437] [default] [JOB 31522] Compacting 1@2 + 24@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:07:43.709933 139957215163968 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23587 Base level 2, inputs: [271352(27MB)], [271378(37MB) 271379(3462KB) 271299(38MB) 271300(34MB) 271301(35MB) 271302(35MB) 271303(35MB) 271304(35MB) 271305(37MB) 271306(35MB) 271307(37MB) 271308(35MB) 271309(34MB) 271310(37MB) 271311(34MB) 271312(37MB) 271313(35MB) 271314(35MB) 271315(36MB) 271316(37MB) 271317(37MB) 271318(6252KB) 271261(36MB) 271380(36MB)] 2024/11/23-17:07:43.717501 139957215163968 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:07:43.717530 139957215163968 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:07:43.717550 139957215163968 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:07:43.717555 139957215163968 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:07:43.717566 139957215163968 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:07:44.514817 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst) /dev/shm/ToplingTemp/Topling-126933-5hQqbj 2024/11/23-17:07:45.244889 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst): first pass time = 0.73's, 183.863'MB/sec 2024/11/23-17:07:46.751316 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst): old prealloc_size = 147639500, real_size = 35739328 2024/11/23-17:07:46.751912 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926185 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 116.91 avg-zval = 36.44 2024/11/23-17:07:46.752720 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:07:47.636278 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271386.sst): second pass time = 1.51's, 71.922'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.884'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3069.842'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 295.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35375.352'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.922'MB/sec zip pipeline throughput = 71.922'MB/sec entries = 926185 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 36.44 usrkeys = 926185 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 36.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926185 bytes = 3125888 } seq expand size = 6483295 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5160 value = 3.2088 dict = 1.00 all = 3.3791 } Zip/UnZip{ index = 0.0951 value = 0.3116 dict = 1.00 all = 0.2959 } ---------------------------- total value len = 8.963388 GB avg = 0.168 KB (by entry num) total key len = 1.466754 GB avg = 0.028 KB total ukey len = 1.016402 GB avg = 0.015 KB total ukey num = 0.066476620 Billion total entry num = 0.053284349 Billion write speed all = 0.352901140 MB/sec (with seq num) write speed all = 0.338478243 MB/sec (without seq num) 2024/11/23-17:07:47.691734 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271386: 926185 keys, 39719938 bytes, temperature: kWarm 2024/11/23-17:07:47.691836 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352867691773, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271386, "file_size": 39719938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4134719764, "table_properties": {"data_size": 33746096, "index_size": 1761472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933180, "raw_average_key_size": 28, "raw_value_size": 108284630, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926185, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352864, "oldest_key_time": 0, "file_creation_time": 1732352864, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271386, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:07:47.733037 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst) /dev/shm/ToplingTemp/Topling-126933-kuUbaj 2024/11/23-17:07:48.938581 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst): first pass time = 1.21's, 111.338'MB/sec 2024/11/23-17:07:50.696689 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst): old prealloc_size = 147639500, real_size = 36444144 2024/11/23-17:07:50.697324 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936261 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0344 GB avg-val = 115.36 avg-zval = 36.75 2024/11/23-17:07:50.698235 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:07:51.510716 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271387.sst): second pass time = 1.76's, 61.477'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 50.282'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2946.992'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.13's, 311.918'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32370.270'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.477'MB/sec zip pipeline throughput = 63.114'MB/sec entries = 936261 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.36 avg-zval = 36.75 usrkeys = 936261 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.31 avg-zval = 36.75 TagRS{ kinds = RS_Key0_TagN bytes = 146400 } TagArray{ size = 929859 bytes = 3486992 } seq expand size = 6509013 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.3778 value = 3.1391 dict = 1.00 all = 3.2792 } Zip/UnZip{ index = 0.0964 value = 0.3186 dict = 1.00 all = 0.3050 } ---------------------------- total value len = 9.077899 GB avg = 0.167 KB (by entry num) total key len = 1.492970 GB avg = 0.028 KB total ukey len = 1.035127 GB avg = 0.015 KB total ukey num = 0.067412881 Billion total entry num = 0.054220610 Billion write speed all = 0.357615713 MB/sec (with seq num) write speed all = 0.342941315 MB/sec (without seq num) 2024/11/23-17:07:51.566568 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271387: 936261 keys, 40930192 bytes, temperature: kWarm 2024/11/23-17:07:51.566642 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352871566605, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271387, "file_size": 40930192, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4134720012, "table_properties": {"data_size": 34405536, "index_size": 1804352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26215308, "raw_average_key_size": 28, "raw_value_size": 108002450, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352867, "oldest_key_time": 0, "file_creation_time": 1732352867, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271387, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:07:51.607178 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst) /dev/shm/ToplingTemp/Topling-126933-14Iovg 2024/11/23-17:07:52.712519 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst): first pass time = 1.11's, 121.432'MB/sec 2024/11/23-17:07:54.605193 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst): old prealloc_size = 147639500, real_size = 34983040 2024/11/23-17:07:54.605838 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081552 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.18 2024/11/23-17:07:54.605878 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:07:55.305832 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271388.sst): second pass time = 1.89's, 54.960'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 51.030'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.748'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.487'MB/sec write SST data time = 0.14's, 247.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.029'MB, throughput = 33220.116'MB/sec zip my value time = 1.89's, unzip length = 0.104'GB zip my value throughput = 54.960'MB/sec zip pipeline throughput = 124.565'MB/sec entries = 1081552 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.18 usrkeys = 1081552 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4474 value = 3.1841 dict = 1.00 all = 3.7541 } Zip/UnZip{ index = 0.0957 value = 0.3141 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 9.181834 GB avg = 0.166 KB (by entry num) total key len = 1.523253 GB avg = 0.028 KB total ukey len = 1.056758 GB avg = 0.015 KB total ukey num = 0.068494433 Billion total entry num = 0.055302162 Billion write speed all = 0.362109853 MB/sec (with seq num) write speed all = 0.347144662 MB/sec (without seq num) 2024/11/23-17:07:55.355245 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271388: 1081552 keys, 35752413 bytes, temperature: kWarm 2024/11/23-17:07:55.355315 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352875355280, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271388, "file_size": 35752413, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32642032, "index_size": 2070464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30283456, "raw_average_key_size": 28, "raw_value_size": 103934397, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081552, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352871, "oldest_key_time": 0, "file_creation_time": 1732352871, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:07:55.394174 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst) /dev/shm/ToplingTemp/Topling-126933-kZILuj 2024/11/23-17:07:56.411332 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst): first pass time = 1.02's, 131.959'MB/sec 2024/11/23-17:07:58.018359 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst): old prealloc_size = 147639500, real_size = 34527408 2024/11/23-17:07:58.018878 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921869 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.59 avg-zval = 35.30 2024/11/23-17:07:58.019471 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:07:58.826809 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271389.sst): second pass time = 1.61's, 67.488'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 46.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.143'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 288.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34817.419'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.488'MB/sec zip pipeline throughput = 67.488'MB/sec entries = 921869 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.59 avg-zval = 35.30 usrkeys = 921869 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.31 avg-zval = 35.30 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 621420 bytes = 2330336 } seq expand size = 4349940 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5129 value = 3.3311 dict = 1.00 all = 3.5447 } Zip/UnZip{ index = 0.0951 value = 0.3002 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 9.294589 GB avg = 0.165 KB (by entry num) total key len = 1.549065 GB avg = 0.028 KB total ukey len = 1.075195 GB avg = 0.015 KB total ukey num = 0.069416302 Billion total entry num = 0.056224031 Billion write speed all = 0.366753357 MB/sec (with seq num) write speed all = 0.351540514 MB/sec (without seq num) 2024/11/23-17:07:58.879405 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271389: 921869 keys, 37864352 bytes, temperature: kWarm 2024/11/23-17:07:58.879516 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352878879461, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271389, "file_size": 37864352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4007408539, "table_properties": {"data_size": 32542944, "index_size": 1753792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812332, "raw_average_key_size": 28, "raw_value_size": 108405398, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352875, "oldest_key_time": 0, "file_creation_time": 1732352875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:07:58.929001 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst) /dev/shm/ToplingTemp/Topling-126933-Q5mCzf 2024/11/23-17:08:00.040750 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst): first pass time = 1.11's, 120.733'MB/sec 2024/11/23-17:08:01.899101 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst): old prealloc_size = 147639500, real_size = 32942896 2024/11/23-17:08:01.899716 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.10 avg-zval = 33.20 2024/11/23-17:08:01.900565 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:02.894963 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271390.sst): second pass time = 1.86's, 58.215'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 37.326'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3032.760'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.307'MB/sec write SST data time = 0.24's, 156.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25319.053'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.215'MB/sec zip pipeline throughput = 129.077'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.10 avg-zval = 33.20 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.10 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4458 value = 3.4966 dict = 1.00 all = 3.6109 } Zip/UnZip{ index = 0.0957 value = 0.2860 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 9.409247 GB avg = 0.165 KB (by entry num) total key len = 1.575145 GB avg = 0.028 KB total ukey len = 1.093824 GB avg = 0.016 KB total ukey num = 0.070347731 Billion total entry num = 0.057155460 Billion write speed all = 0.371462271 MB/sec (with seq num) write speed all = 0.355999533 MB/sec (without seq num) 2024/11/23-17:08:02.949966 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271390: 931429 keys, 37170352 bytes, temperature: kWarm 2024/11/23-17:08:02.950076 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352882950025, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271390, "file_size": 37170352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30926496, "index_size": 1783360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352878, "oldest_key_time": 0, "file_creation_time": 1732352878, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:03.030392 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst) /dev/shm/ToplingTemp/Topling-126933-ApcFZe 2024/11/23-17:08:04.007087 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst): first pass time = 0.98's, 137.429'MB/sec 2024/11/23-17:08:05.573586 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst): old prealloc_size = 147639500, real_size = 32890896 2024/11/23-17:08:05.574161 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.12 avg-zval = 33.16 2024/11/23-17:08:05.574936 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:06.642368 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271391.sst): second pass time = 1.57's, 69.069'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.621'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.507'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.248'MB/sec write SST data time = 0.13's, 286.374'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35950.655'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.069'MB/sec zip pipeline throughput = 93.206'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 33.16 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 33.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5120 value = 3.5013 dict = 1.00 all = 3.6273 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 9.523907 GB avg = 0.164 KB (by entry num) total key len = 1.601222 GB avg = 0.028 KB total ukey len = 1.112450 GB avg = 0.016 KB total ukey num = 0.071279040 Billion total entry num = 0.058086769 Billion write speed all = 0.376173941 MB/sec (with seq num) write speed all = 0.360461239 MB/sec (without seq num) 2024/11/23-17:08:06.693559 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271391: 931309 keys, 37001740 bytes, temperature: kWarm 2024/11/23-17:08:06.693628 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352886693593, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271391, "file_size": 37001740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30885952, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352883, "oldest_key_time": 0, "file_creation_time": 1732352883, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:06.742714 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst) /dev/shm/ToplingTemp/Topling-126933-8miKEf 2024/11/23-17:08:07.657930 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst): first pass time = 0.92's, 146.659'MB/sec 2024/11/23-17:08:08.990148 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst): old prealloc_size = 147639500, real_size = 32978272 2024/11/23-17:08:08.990711 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0310 GB avg-val = 116.09 avg-zval = 33.25 2024/11/23-17:08:08.991542 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:09.852919 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271392.sst): second pass time = 1.33's, 81.224'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.259'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.748'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 285.107'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28845.880'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.224'MB/sec zip pipeline throughput = 81.224'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.25 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5133 value = 3.4913 dict = 1.00 all = 3.5962 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 9.638564 GB avg = 0.163 KB (by entry num) total key len = 1.627303 GB avg = 0.028 KB total ukey len = 1.131079 GB avg = 0.016 KB total ukey num = 0.072210502 Billion total entry num = 0.059018231 Billion write speed all = 0.380891366 MB/sec (with seq num) write speed all = 0.364928433 MB/sec (without seq num) 2024/11/23-17:08:09.901872 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271392: 931462 keys, 37322124 bytes, temperature: kWarm 2024/11/23-17:08:09.901944 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352889901908, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271392, "file_size": 37322124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30973264, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352886, "oldest_key_time": 0, "file_creation_time": 1732352886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:09.941273 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst) /dev/shm/ToplingTemp/Topling-126933-fvrcgf 2024/11/23-17:08:10.929474 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst): first pass time = 0.99's, 135.825'MB/sec 2024/11/23-17:08:12.260588 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst): old prealloc_size = 147639500, real_size = 34484768 2024/11/23-17:08:12.261140 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927853 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.65 avg-zval = 34.99 2024/11/23-17:08:12.263053 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:13.112211 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271393.sst): second pass time = 1.33's, 81.365'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.447'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.709'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 295.691'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25144.959'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.365'MB/sec zip pipeline throughput = 81.365'MB/sec entries = 927853 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.65 avg-zval = 34.99 usrkeys = 927853 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.65 avg-zval = 34.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927853 bytes = 3363488 } seq expand size = 6494971 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.3766 value = 3.3341 dict = 1.00 all = 3.4680 } Zip/UnZip{ index = 0.0964 value = 0.2999 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 9.753297 GB avg = 0.163 KB (by entry num) total key len = 1.653283 GB avg = 0.028 KB total ukey len = 1.149636 GB avg = 0.016 KB total ukey num = 0.073138355 Billion total entry num = 0.059946084 Billion write speed all = 0.385606276 MB/sec (with seq num) write speed all = 0.369394169 MB/sec (without seq num) 2024/11/23-17:08:13.163696 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271393: 927853 keys, 38702136 bytes, temperature: kWarm 2024/11/23-17:08:13.163766 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352893163731, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271393, "file_size": 38702136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775967, "table_properties": {"data_size": 32464272, "index_size": 1788352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979884, "raw_average_key_size": 28, "raw_value_size": 108237858, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927853, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352889, "oldest_key_time": 0, "file_creation_time": 1732352889, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:13.204113 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst) /dev/shm/ToplingTemp/Topling-126933-EoDIPg 2024/11/23-17:08:14.235101 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst): first pass time = 1.03's, 130.189'MB/sec 2024/11/23-17:08:15.581950 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst): old prealloc_size = 147639500, real_size = 34433104 2024/11/23-17:08:15.582522 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925453 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.03 avg-zval = 35.05 2024/11/23-17:08:15.583324 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:16.429137 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271394.sst): second pass time = 1.35's, 80.460'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.615'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3358.748'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 300.369'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34675.350'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.460'MB/sec zip pipeline throughput = 80.460'MB/sec entries = 925453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 35.05 usrkeys = 925453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 35.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925453 bytes = 3354784 } seq expand size = 6478171 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5146 value = 3.3385 dict = 1.00 all = 3.4733 } Zip/UnZip{ index = 0.0951 value = 0.2995 dict = 1.00 all = 0.2879 } ---------------------------- total value len = 9.868080 GB avg = 0.162 KB (by entry num) total key len = 1.679196 GB avg = 0.028 KB total ukey len = 1.168145 GB avg = 0.016 KB total ukey num = 0.074063808 Billion total entry num = 0.060871537 Billion write speed all = 0.390318823 MB/sec (with seq num) write speed all = 0.373858278 MB/sec (without seq num) 2024/11/23-17:08:16.479827 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271394: 925453 keys, 38643220 bytes, temperature: kWarm 2024/11/23-17:08:16.479898 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352896479862, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271394, "file_size": 38643220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 32441216, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912684, "raw_average_key_size": 28, "raw_value_size": 108305174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352893, "oldest_key_time": 0, "file_creation_time": 1732352893, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:16.522159 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst) /dev/shm/ToplingTemp/Topling-126933-xZ9HDf 2024/11/23-17:08:17.550232 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst): first pass time = 1.03's, 130.558'MB/sec 2024/11/23-17:08:18.900224 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst): old prealloc_size = 147639500, real_size = 36133408 2024/11/23-17:08:18.900806 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926995 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 116.79 avg-zval = 36.81 2024/11/23-17:08:18.902722 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:19.707143 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271395.sst): second pass time = 1.35's, 80.245'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.481'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.128'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.343'MB/sec write SST data time = 0.13's, 303.995'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35199.074'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.245'MB/sec zip pipeline throughput = 80.245'MB/sec entries = 926995 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 36.81 usrkeys = 926995 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.41 avg-zval = 36.81 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 743998 bytes = 2790000 } seq expand size = 5207986 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4455 value = 3.1724 dict = 1.00 all = 3.3617 } Zip/UnZip{ index = 0.0957 value = 0.3152 dict = 1.00 all = 0.2975 } ---------------------------- total value len = 9.981550 GB avg = 0.162 KB (by entry num) total key len = 1.705151 GB avg = 0.028 KB total ukey len = 1.186685 GB avg = 0.016 KB total ukey num = 0.074990803 Billion total entry num = 0.061798532 Billion write speed all = 0.394987901 MB/sec (with seq num) write speed all = 0.378278535 MB/sec (without seq num) 2024/11/23-17:08:19.758436 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271395: 926995 keys, 39926074 bytes, temperature: kWarm 2024/11/23-17:08:19.758512 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352899758477, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271395, "file_size": 39926074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34126544, "index_size": 1774912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955860, "raw_average_key_size": 28, "raw_value_size": 108261926, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926995, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352896, "oldest_key_time": 0, "file_creation_time": 1732352896, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:19.798098 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst) /dev/shm/ToplingTemp/Topling-126933-v5beof 2024/11/23-17:08:20.819420 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst): first pass time = 1.02's, 131.421'MB/sec 2024/11/23-17:08:22.143402 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst): old prealloc_size = 147639500, real_size = 33055664 2024/11/23-17:08:22.143938 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930230 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.28 avg-zval = 33.38 2024/11/23-17:08:22.144763 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:23.001505 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271396.sst): second pass time = 1.32's, 81.749'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3465.900'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.388'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33374.722'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.749'MB/sec zip pipeline throughput = 81.749'MB/sec entries = 930230 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 33.38 usrkeys = 930230 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930230 bytes = 3372096 } seq expand size = 6511610 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5138 value = 3.4834 dict = 1.00 all = 3.6003 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 10.096233 GB avg = 0.161 KB (by entry num) total key len = 1.731198 GB avg = 0.028 KB total ukey len = 1.205290 GB avg = 0.016 KB total ukey num = 0.075921033 Billion total entry num = 0.062728762 Billion write speed all = 0.399699781 MB/sec (with seq num) write speed all = 0.382740783 MB/sec (without seq num) 2024/11/23-17:08:23.049304 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271396: 930230 keys, 37280042 bytes, temperature: kWarm 2024/11/23-17:08:23.049389 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352903049341, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271396, "file_size": 37280042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 31053408, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046440, "raw_average_key_size": 28, "raw_value_size": 108171412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930230, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352899, "oldest_key_time": 0, "file_creation_time": 1732352899, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:23.084005 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst) /dev/shm/ToplingTemp/Topling-126933-7S4ibg 2024/11/23-17:08:23.954816 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst): first pass time = 0.87's, 154.136'MB/sec 2024/11/23-17:08:25.275730 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst): old prealloc_size = 147639500, real_size = 32151776 2024/11/23-17:08:25.276283 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936596 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0301 GB avg-val = 115.30 avg-zval = 32.18 2024/11/23-17:08:25.277104 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:26.139572 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271397.sst): second pass time = 1.32's, 81.805'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3368.631'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 283.331'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32991.144'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.805'MB/sec zip pipeline throughput = 81.805'MB/sec entries = 936596 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.30 avg-zval = 32.18 usrkeys = 936596 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.30 avg-zval = 32.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936596 bytes = 3395168 } seq expand size = 6556172 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5124 value = 3.5836 dict = 1.00 all = 3.6878 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 10.210782 GB avg = 0.160 KB (by entry num) total key len = 1.757423 GB avg = 0.028 KB total ukey len = 1.224022 GB avg = 0.016 KB total ukey num = 0.076857629 Billion total entry num = 0.063665358 Billion write speed all = 0.404414252 MB/sec (with seq num) write speed all = 0.387203866 MB/sec (without seq num) 2024/11/23-17:08:26.185471 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271397: 936596 keys, 36395272 bytes, temperature: kWarm 2024/11/23-17:08:26.185540 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352906185505, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271397, "file_size": 36395272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30135568, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26224688, "raw_average_key_size": 28, "raw_value_size": 107993132, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936596, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352903, "oldest_key_time": 0, "file_creation_time": 1732352903, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:26.222070 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst) /dev/shm/ToplingTemp/Topling-126933-2OJnIh 2024/11/23-17:08:27.276570 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst): first pass time = 1.05's, 127.287'MB/sec 2024/11/23-17:08:28.639447 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst): old prealloc_size = 147639500, real_size = 35238992 2024/11/23-17:08:28.640006 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927911 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.65 avg-zval = 35.81 2024/11/23-17:08:28.640820 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:29.476384 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271398.sst): second pass time = 1.36's, 79.462'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.284'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3424.006'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 309.341'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33409.435'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.462'MB/sec zip pipeline throughput = 79.462'MB/sec entries = 927911 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.65 avg-zval = 35.81 usrkeys = 927911 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.65 avg-zval = 35.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927911 bytes = 3479680 } seq expand size = 6495377 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4438 value = 3.2572 dict = 1.00 all = 3.3917 } Zip/UnZip{ index = 0.0958 value = 0.3070 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 10.325514 GB avg = 0.160 KB (by entry num) total key len = 1.783404 GB avg = 0.028 KB total ukey len = 1.242580 GB avg = 0.016 KB total ukey num = 0.077785540 Billion total entry num = 0.064593269 Billion write speed all = 0.409122920 MB/sec (with seq num) write speed all = 0.391663664 MB/sec (without seq num) 2024/11/23-17:08:29.526644 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271398: 927911 keys, 39572520 bytes, temperature: kWarm 2024/11/23-17:08:29.526715 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352909526679, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271398, "file_size": 39572520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33229888, "index_size": 1776960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981508, "raw_average_key_size": 28, "raw_value_size": 108236234, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927911, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352906, "oldest_key_time": 0, "file_creation_time": 1732352906, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:29.563743 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst) /dev/shm/ToplingTemp/Topling-126933-B3Ovvi 2024/11/23-17:08:30.545538 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst): first pass time = 0.98's, 136.712'MB/sec 2024/11/23-17:08:31.847581 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst): old prealloc_size = 147639500, real_size = 31547488 2024/11/23-17:08:31.848130 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 116.13 avg-zval = 31.72 2024/11/23-17:08:31.848920 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:32.688191 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271399.sst): second pass time = 1.30's, 83.140'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.560'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.409'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 278.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33463.469'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.140'MB/sec zip pipeline throughput = 83.140'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 31.72 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 31.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5123 value = 3.6606 dict = 1.00 all = 3.7517 } Zip/UnZip{ index = 0.0951 value = 0.2732 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 10.440176 GB avg = 0.159 KB (by entry num) total key len = 1.809479 GB avg = 0.028 KB total ukey len = 1.261205 GB avg = 0.016 KB total ukey num = 0.078716782 Billion total entry num = 0.065524511 Billion write speed all = 0.413833061 MB/sec (with seq num) write speed all = 0.396124017 MB/sec (without seq num) 2024/11/23-17:08:32.733381 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271399: 931242 keys, 35774900 bytes, temperature: kWarm 2024/11/23-17:08:32.733460 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352912733417, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271399, "file_size": 35774900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 29542800, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352909, "oldest_key_time": 0, "file_creation_time": 1732352909, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:32.771201 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst) /dev/shm/ToplingTemp/Topling-126933-9LNr6g 2024/11/23-17:08:33.678219 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst): first pass time = 0.91's, 147.983'MB/sec 2024/11/23-17:08:35.017031 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst): old prealloc_size = 147639500, real_size = 35293104 2024/11/23-17:08:35.017614 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.46 avg-zval = 36.10 2024/11/23-17:08:35.019552 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:35.860544 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271400.sst): second pass time = 1.34's, 81.005'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.962'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.640'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 305.324'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 37619.769'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.005'MB/sec zip pipeline throughput = 81.005'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.10 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5131 value = 3.2540 dict = 1.00 all = 3.3883 } Zip/UnZip{ index = 0.0951 value = 0.3073 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 10.555017 GB avg = 0.159 KB (by entry num) total key len = 1.835314 GB avg = 0.028 KB total ukey len = 1.279659 GB avg = 0.016 KB total ukey num = 0.079639479 Billion total entry num = 0.066447208 Billion write speed all = 0.418540719 MB/sec (with seq num) write speed all = 0.400584226 MB/sec (without seq num) 2024/11/23-17:08:35.909579 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271400: 922697 keys, 39611586 bytes, temperature: kWarm 2024/11/23-17:08:35.909649 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352915909614, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271400, "file_size": 39611586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33306912, "index_size": 1755328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352912, "oldest_key_time": 0, "file_creation_time": 1732352912, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271400, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:35.948961 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst) /dev/shm/ToplingTemp/Topling-126933-q3di0i 2024/11/23-17:08:36.854786 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst): first pass time = 0.91's, 148.182'MB/sec 2024/11/23-17:08:38.185130 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst): old prealloc_size = 147639500, real_size = 32685104 2024/11/23-17:08:38.185671 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.15 avg-zval = 32.94 2024/11/23-17:08:38.186498 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:39.048096 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271401.sst): second pass time = 1.33's, 81.339'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.910'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3439.960'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 286.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 29920.160'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.339'MB/sec zip pipeline throughput = 81.339'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 32.94 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4441 value = 3.5263 dict = 1.00 all = 3.6247 } Zip/UnZip{ index = 0.0957 value = 0.2836 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 10.669682 GB avg = 0.158 KB (by entry num) total key len = 1.861385 GB avg = 0.028 KB total ukey len = 1.298281 GB avg = 0.016 KB total ukey num = 0.080570562 Billion total entry num = 0.067378291 Billion write speed all = 0.423249136 MB/sec (with seq num) write speed all = 0.405042990 MB/sec (without seq num) 2024/11/23-17:08:39.095115 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271401: 931083 keys, 37028372 bytes, temperature: kWarm 2024/11/23-17:08:39.095188 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352919095151, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271401, "file_size": 37028372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30669152, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352915, "oldest_key_time": 0, "file_creation_time": 1732352915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271401, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:39.130532 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst) /dev/shm/ToplingTemp/Topling-126933-pH3XTg 2024/11/23-17:08:39.970875 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst): first pass time = 0.84's, 159.725'MB/sec 2024/11/23-17:08:41.303579 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst): old prealloc_size = 147639500, real_size = 33094272 2024/11/23-17:08:41.304131 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926724 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.83 avg-zval = 33.55 2024/11/23-17:08:41.304787 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:42.123930 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271402.sst): second pass time = 1.33's, 81.287'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.649'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3359.386'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 273.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25868.483'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.287'MB/sec zip pipeline throughput = 81.287'MB/sec entries = 926724 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.83 avg-zval = 33.55 usrkeys = 926724 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.10 avg-zval = 33.55 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 697606 bytes = 2528832 } seq expand size = 4883242 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4432 value = 3.4827 dict = 1.00 all = 3.6644 } Zip/UnZip{ index = 0.0958 value = 0.2871 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 10.782835 GB avg = 0.158 KB (by entry num) total key len = 1.887333 GB avg = 0.028 KB total ukey len = 1.316815 GB avg = 0.016 KB total ukey num = 0.081497286 Billion total entry num = 0.068305015 Billion write speed all = 0.427902957 MB/sec (with seq num) write speed all = 0.409448320 MB/sec (without seq num) 2024/11/23-17:08:42.170545 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271402: 926724 keys, 36627010 bytes, temperature: kWarm 2024/11/23-17:08:42.170619 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352922170582, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271402, "file_size": 36627010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 31087600, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948272, "raw_average_key_size": 28, "raw_value_size": 108269532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352919, "oldest_key_time": 0, "file_creation_time": 1732352919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271402, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:42.209080 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst) /dev/shm/ToplingTemp/Topling-126933-QIZ3Bf 2024/11/23-17:08:43.096082 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst): first pass time = 0.89's, 151.324'MB/sec 2024/11/23-17:08:44.581729 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst): old prealloc_size = 147639500, real_size = 36837120 2024/11/23-17:08:44.582257 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926870 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.81 avg-zval = 37.59 2024/11/23-17:08:44.582493 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:45.298333 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271403.sst): second pass time = 1.48's, 72.933'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.480'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.549'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.393'MB/sec write SST data time = 0.13's, 301.756'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30715.159'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.933'MB/sec zip pipeline throughput = 72.933'MB/sec entries = 926870 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 37.59 usrkeys = 926870 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.97 avg-zval = 37.59 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5135 value = 3.1073 dict = 1.00 all = 3.5023 } Zip/UnZip{ index = 0.0951 value = 0.3218 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 10.892181 GB avg = 0.157 KB (by entry num) total key len = 1.913285 GB avg = 0.028 KB total ukey len = 1.335352 GB avg = 0.016 KB total ukey num = 0.082424156 Billion total entry num = 0.069231885 Billion write speed all = 0.432425967 MB/sec (with seq num) write speed all = 0.413722913 MB/sec (without seq num) 2024/11/23-17:08:45.347038 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271403: 926870 keys, 38322543 bytes, temperature: kWarm 2024/11/23-17:08:45.347109 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352925347073, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271403, "file_size": 38322543, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34842032, "index_size": 1763200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952360, "raw_average_key_size": 28, "raw_value_size": 108265468, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926870, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352922, "oldest_key_time": 0, "file_creation_time": 1732352922, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271403, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:45.390779 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst) /dev/shm/ToplingTemp/Topling-126933-lnFVrj 2024/11/23-17:08:46.244579 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst): first pass time = 0.85's, 157.208'MB/sec 2024/11/23-17:08:47.614058 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst): old prealloc_size = 147639500, real_size = 35617200 2024/11/23-17:08:47.614590 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924776 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 117.14 avg-zval = 36.36 2024/11/23-17:08:47.615149 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:48.411009 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271404.sst): second pass time = 1.37's, 79.146'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 48.600'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3459.925'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.347'MB/sec write SST data time = 0.13's, 295.557'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32422.732'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.146'MB/sec zip pipeline throughput = 79.146'MB/sec entries = 924776 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 36.36 usrkeys = 924776 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.86 avg-zval = 36.36 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 624773 bytes = 2108624 } seq expand size = 4373411 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5123 value = 3.2214 dict = 1.00 all = 3.4655 } Zip/UnZip{ index = 0.0951 value = 0.3104 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 11.004879 GB avg = 0.157 KB (by entry num) total key len = 1.939179 GB avg = 0.028 KB total ukey len = 1.353848 GB avg = 0.016 KB total ukey num = 0.083348932 Billion total entry num = 0.070156661 Billion write speed all = 0.437060095 MB/sec (with seq num) write speed all = 0.418109204 MB/sec (without seq num) 2024/11/23-17:08:48.460445 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271404: 924776 keys, 38729920 bytes, temperature: kWarm 2024/11/23-17:08:48.460525 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352928460486, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271404, "file_size": 38729920, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4134719850, "table_properties": {"data_size": 33626400, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893728, "raw_average_key_size": 28, "raw_value_size": 108324104, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924776, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352925, "oldest_key_time": 0, "file_creation_time": 1732352925, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271404, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:48.500467 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst) /dev/shm/ToplingTemp/Topling-126933-0HyJ4h 2024/11/23-17:08:49.523246 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst): first pass time = 1.02's, 131.234'MB/sec 2024/11/23-17:08:50.876048 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst): old prealloc_size = 147639500, real_size = 35570320 2024/11/23-17:08:50.876603 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927974 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.64 avg-zval = 36.18 2024/11/23-17:08:50.877333 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:51.709673 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271405.sst): second pass time = 1.35's, 80.055'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.312'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.269'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 302.076'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32578.223'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.055'MB/sec zip pipeline throughput = 80.055'MB/sec entries = 927974 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 36.18 usrkeys = 927974 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 36.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927974 bytes = 3015936 } seq expand size = 6495818 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5138 value = 3.2239 dict = 1.00 all = 3.4031 } Zip/UnZip{ index = 0.0951 value = 0.3102 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 11.119609 GB avg = 0.156 KB (by entry num) total key len = 1.965162 GB avg = 0.028 KB total ukey len = 1.372407 GB avg = 0.016 KB total ukey num = 0.084276906 Billion total entry num = 0.071084635 Billion write speed all = 0.441762132 MB/sec (with seq num) write speed all = 0.422562713 MB/sec (without seq num) 2024/11/23-17:08:51.757352 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271405: 927974 keys, 39439592 bytes, temperature: kWarm 2024/11/23-17:08:51.757423 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352931757387, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271405, "file_size": 39439592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4134719922, "table_properties": {"data_size": 33572864, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983272, "raw_average_key_size": 28, "raw_value_size": 108234588, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927974, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352928, "oldest_key_time": 0, "file_creation_time": 1732352928, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271405, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:51.800631 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst) /dev/shm/ToplingTemp/Topling-126933-CUyHtj 2024/11/23-17:08:52.915010 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst): first pass time = 1.11's, 120.447'MB/sec 2024/11/23-17:08:54.266904 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst): old prealloc_size = 147639500, real_size = 35891296 2024/11/23-17:08:54.267465 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928688 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.52 avg-zval = 36.49 2024/11/23-17:08:54.268206 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:55.100328 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271406.sst): second pass time = 1.35's, 80.096'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.851'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.672'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 304.615'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31325.578'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.096'MB/sec zip pipeline throughput = 80.096'MB/sec entries = 928688 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 36.49 usrkeys = 928688 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 36.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928688 bytes = 3134336 } seq expand size = 6500816 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.5139 value = 3.1929 dict = 1.00 all = 3.3656 } Zip/UnZip{ index = 0.0951 value = 0.3132 dict = 1.00 all = 0.2971 } ---------------------------- total value len = 11.234324 GB avg = 0.156 KB (by entry num) total key len = 1.991166 GB avg = 0.028 KB total ukey len = 1.390981 GB avg = 0.016 KB total ukey num = 0.085205594 Billion total entry num = 0.072013323 Billion write speed all = 0.446461903 MB/sec (with seq num) write speed all = 0.427013880 MB/sec (without seq num) 2024/11/23-17:08:55.149610 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271406: 928688 keys, 39879136 bytes, temperature: kWarm 2024/11/23-17:08:55.149680 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352935149645, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271406, "file_size": 39879136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4134720173, "table_properties": {"data_size": 33892368, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003264, "raw_average_key_size": 28, "raw_value_size": 108214584, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928688, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352931, "oldest_key_time": 0, "file_creation_time": 1732352931, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271406, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:55.188055 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst) /dev/shm/ToplingTemp/Topling-126933-xSdYBi 2024/11/23-17:08:56.235017 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst): first pass time = 1.05's, 128.201'MB/sec 2024/11/23-17:08:57.562183 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst): old prealloc_size = 147639500, real_size = 34614896 2024/11/23-17:08:57.562736 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928223 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0326 GB avg-val = 116.60 avg-zval = 35.14 2024/11/23-17:08:57.563462 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:08:58.408629 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271407.sst): second pass time = 1.33's, 81.628'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.836'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 293.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33303.141'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.628'MB/sec zip pipeline throughput = 81.628'MB/sec entries = 928223 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 35.14 usrkeys = 928223 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 35.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928223 bytes = 3016736 } seq expand size = 6497561 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5132 value = 3.3181 dict = 1.00 all = 3.4872 } Zip/UnZip{ index = 0.0951 value = 0.3014 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 11.349050 GB avg = 0.156 KB (by entry num) total key len = 2.017156 GB avg = 0.028 KB total ukey len = 1.409546 GB avg = 0.016 KB total ukey num = 0.086133817 Billion total entry num = 0.072941546 Billion write speed all = 0.451161744 MB/sec (with seq num) write speed all = 0.431465242 MB/sec (without seq num) 2024/11/23-17:08:58.455778 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271407: 928223 keys, 38488656 bytes, temperature: kWarm 2024/11/23-17:08:58.455850 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352938455813, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271407, "file_size": 38488656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4134719597, "table_properties": {"data_size": 32616864, "index_size": 1765824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990244, "raw_average_key_size": 28, "raw_value_size": 108227606, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928223, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352935, "oldest_key_time": 0, "file_creation_time": 1732352935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271407, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:58.498229 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst) /dev/shm/ToplingTemp/Topling-126933-SerEYf 2024/11/23-17:08:58.712484 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst): first pass time = 0.21's, 240.877'MB/sec 2024/11/23-17:08:59.221689 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst): old prealloc_size = 147639500, real_size = 14020448 2024/11/23-17:08:59.221954 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 358910 prefix = 4 raw-key = 0.0057 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0415 GB zip-val = 0.0132 GB avg-val = 115.76 avg-zval = 36.91 2024/11/23-17:08:59.222318 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.052 GB 2024/11/23-17:08:59.543954 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00/271408.sst): second pass time = 0.51's, 81.708'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.32's, 47.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2865.848'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.330'MB/sec write SST data time = 0.05's, 307.225'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.420'MB, throughput = 14368.886'MB/sec zip my value time = 0.51's, unzip length = 0.042'GB zip my value throughput = 81.708'MB/sec zip pipeline throughput = 81.708'MB/sec entries = 358910 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.76 avg-zval = 36.91 usrkeys = 358910 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.76 avg-zval = 36.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 358910 bytes = 1076752 } seq expand size = 2512370 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0072 GB value = 0.0415 GB dict = 0.42 MB all = 0.0516 GB } __ZipSize{ index = 0.0007 GB value = 0.0132 GB dict = 0.42 MB all = 0.0154 GB } UnZip/Zip{ index = 10.5028 value = 3.1365 dict = 1.00 all = 3.3439 } Zip/UnZip{ index = 0.0952 value = 0.3188 dict = 1.00 all = 0.2990 } ---------------------------- total value len = 11.393111 GB avg = 0.155 KB (by entry num) total key len = 2.027205 GB avg = 0.028 KB total ukey len = 1.416724 GB avg = 0.016 KB total ukey num = 0.086492727 Billion total entry num = 0.073300456 Billion write speed all = 0.452970842 MB/sec (with seq num) write speed all = 0.433178182 MB/sec (without seq num) 2024/11/23-17:08:59.568310 139957215163968 [compaction_job.cc:1948] [default] [JOB 31522] Generated table #271408: 358910 keys, 15430507 bytes, temperature: kWarm 2024/11/23-17:08:59.568378 139957215163968 EVENT_LOG_v1 {"time_micros": 1732352939568344, "cf_name": "default", "job": 31522, "event": "table_file_creation", "file_number": 271408, "file_size": 15430507, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223949, "largest_seqno": 4134720208, "table_properties": {"data_size": 13247056, "index_size": 683456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10049480, "raw_average_key_size": 28, "raw_value_size": 41549116, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 358910, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352938, "oldest_key_time": 0, "file_creation_time": 1732352938, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271408, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:08:59.585619 139957215163968 [compaction_job.cc:703] job-31522: subcompact[0], size: 0.855613 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31522/att-00 [271386,271387,271388,271389,271390,271391,271392,271393,271394,271395,271396,271397,271398,271399,271400,271401,271402,271403,271404,271405,271406,271407,271408] 2024/11/23-17:08:59.724602 139957215163968 (Original Log Time 2024/11/23-17:07:43.710012) EVENT_LOG_v1 {"time_micros": 1732352863709949, "job": 31522, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [271352], "files_L3": [271378, 271379, 271299, 271300, 271301, 271302, 271303, 271304, 271305, 271306, 271307, 271308, 271309, 271310, 271311, 271312, 271313, 271314, 271315, 271316, 271317, 271318, 271261, 271380], "score": 0.998536, "input_data_size": 875092349, "oldest_snapshot_seqno": 4134984140} 2024/11/23-17:08:59.724606 139957215163968 (Original Log Time 2024/11/23-17:08:59.722866) [compaction_job.cc:2022] [default] [JOB 31522] Compacted 1@2 + 24@3 files to L3 => 855612597 bytes 2024/11/23-17:08:59.724608 139957215163968 (Original Log Time 2024/11/23-17:08:59.724524) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.3 wr, level 3, files in(1, 24) out(23 +0 blob) MB in(27.1, 807.5 +0.0 blob) out(816.0 +0.0 blob), read-write-amplify(60.9) write-amplify(30.1) OK, records in: 21035675, records dropped: 93383 output_compression: Snappy 2024/11/23-17:08:59.724611 139957215163968 (Original Log Time 2024/11/23-17:08:59.724574) EVENT_LOG_v1 {"time_micros": 1732352939724551, "cf": "default", "job": 31522, "event": "compaction_finished", "compaction_time_micros": 75875616, "compaction_time_cpu_micros": 31566865, "output_level": 3, "num_output_files": 23, "total_output_size": 855612597, "num_input_records": 21035675, "num_output_records": 20942292, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 591984540, "file_range_sync_nanos": 0, "file_fsync_nanos": 17016, "file_prepare_write_nanos": 62196, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:08:59.848339 139957215163968 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732352863,"executesMs":76164,"compactionJobId":31522,"attempt":0,"compactionInputRawBytes":3017650719,"compactionInputZipBytes":875092349,"compactionOutputRawBytes":0,"compactionOutputZipBytes":855612597,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5a1344e39b26343e873ed2ed72a1f10f-5814a1831f5405b4-00"}}, response = 2024/11/23-17:08:59.848429 139957215163968 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31522/att-00: olev 3, work 76.164 s, result 7.837 ms, install 1.763 ms, input{raw 2.810 GiB zip 834.553 MiB}