2024/11/24-01:51:10.442799 139957609621056 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00: fork to child time = 0.007692 sec 2024/11/24-01:51:10.443536 139957609621056 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00 2024/11/24-01:51:10.449349 139957609621056 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00 2024/11/24-01:51:10.450347 139957609621056 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:51:10.450605 139957609621056 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:51:10.500358 139957609621056 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39641/att-00/MANIFEST-206347 2024/11/24-01:51:10.532867 139957609621056 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39641/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 287856, last_sequence is 4381418442, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287688 2024/11/24-01:51:10.532873 139957609621056 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:51:10.533061 139957609621056 [version_set.cc:5721] Creating manifest 287856 2024/11/24-01:51:10.570758 139957609621056 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:51:10.590330 139957609621056 [compaction_job.cc:2437] [default] [JOB 39641] Compacting 1@2 + 16@3 files to L3, score 0.54, subcompactions 1 : 1 2024/11/24-01:51:10.590344 139957609621056 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29701 Base level 2, inputs: [287814(27MB)], [287854(5816KB) 287762(34MB) 287763(33MB) 287764(33MB) 287765(32MB) 287766(35MB) 287767(32MB) 287768(33MB) 287769(35MB) 287770(33MB) 287771(33MB) 287779(35MB) 287780(34MB) 287781(33MB) 287782(35MB) 287835(34MB)] 2024/11/24-01:51:10.598800 139957609621056 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.598831 139957609621056 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.598851 139957609621056 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.598857 139957609621056 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:51:10.598867 139957609621056 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:51:10.888843 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst) /dev/shm/ToplingTemp/Topling-241780-2OV8Tr 2024/11/24-01:51:11.808778 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst): first pass time = 0.92's, 145.906'MB/sec 2024/11/24-01:51:13.368492 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst): old prealloc_size = 147639500, real_size = 32992448 2024/11/24-01:51:13.369056 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925883 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.96 avg-zval = 33.48 2024/11/24-01:51:13.369935 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:14.218973 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287857.sst): second pass time = 1.56's, 69.470'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3242.752'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 295.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28547.006'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.470'MB/sec zip pipeline throughput = 115.710'MB/sec entries = 925883 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.48 usrkeys = 925883 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925883 bytes = 3124864 } seq expand size = 6481181 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5157 value = 3.4933 dict = 1.00 all = 3.6302 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 8.963394 GB avg = 0.168 KB (by entry num) total key len = 1.466746 GB avg = 0.028 KB total ukey len = 1.016396 GB avg = 0.015 KB total ukey num = 0.066476318 Billion total entry num = 0.053284047 Billion write speed all = 0.171092470 MB/sec (with seq num) write speed all = 0.164100043 MB/sec (without seq num) 2024/11/24-01:51:14.264673 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287857: 925883 keys, 36972118 bytes, temperature: kWarm 2024/11/24-01:51:14.264773 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384274264714, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287857, "file_size": 36972118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4381223891, "table_properties": {"data_size": 30999856, "index_size": 1760960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924724, "raw_average_key_size": 28, "raw_value_size": 108293138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925883, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384270, "oldest_key_time": 0, "file_creation_time": 1732384270, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:14.300816 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst) /dev/shm/ToplingTemp/Topling-241780-txwAhu 2024/11/24-01:51:15.262394 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst): first pass time = 0.96's, 139.586'MB/sec 2024/11/24-01:51:16.704444 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst): old prealloc_size = 147639500, real_size = 32550528 2024/11/24-01:51:16.705073 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932326 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.96 avg-zval = 32.75 2024/11/24-01:51:16.705832 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:17.565120 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287858.sst): second pass time = 1.44's, 75.016'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.470'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2962.139'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 277.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33339.725'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.016'MB/sec zip pipeline throughput = 80.576'MB/sec entries = 932326 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.96 avg-zval = 32.75 usrkeys = 932326 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.96 avg-zval = 32.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932326 bytes = 3146608 } seq expand size = 6526282 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4450 value = 3.5410 dict = 1.00 all = 3.6720 } Zip/UnZip{ index = 0.0957 value = 0.2824 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 9.078033 GB avg = 0.167 KB (by entry num) total key len = 1.492851 GB avg = 0.028 KB total ukey len = 1.035042 GB avg = 0.015 KB total ukey num = 0.067408644 Billion total entry num = 0.054216373 Billion write speed all = 0.173391670 MB/sec (with seq num) write speed all = 0.166277285 MB/sec (without seq num) 2024/11/24-01:51:17.611681 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287858: 932326 keys, 36551556 bytes, temperature: kWarm 2024/11/24-01:51:17.611768 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384277611715, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287858, "file_size": 36551556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4381223092, "table_properties": {"data_size": 30532016, "index_size": 1785216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105128, "raw_average_key_size": 28, "raw_value_size": 108112608, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384274, "oldest_key_time": 0, "file_creation_time": 1732384274, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:17.649784 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst) /dev/shm/ToplingTemp/Topling-241780-nPXbLs 2024/11/24-01:51:18.537449 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst): first pass time = 0.89's, 151.210'MB/sec 2024/11/24-01:51:20.022016 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst): old prealloc_size = 147639500, real_size = 29106832 2024/11/24-01:51:20.022550 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 942880 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0271 GB avg-val = 114.35 avg-zval = 28.72 2024/11/24-01:51:20.023317 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:20.860925 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287859.sst): second pass time = 1.48's, 72.659'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.834'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 253.417'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 27136.791'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.659'MB/sec zip pipeline throughput = 122.390'MB/sec entries = 942880 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.35 avg-zval = 28.72 usrkeys = 942880 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.35 avg-zval = 28.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 942880 bytes = 3182240 } seq expand size = 6600160 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0271 GB dict = 1.07 MB all = 0.0331 GB } UnZip/Zip{ index = 10.5116 value = 3.9819 dict = 1.00 all = 4.0507 } Zip/UnZip{ index = 0.0951 value = 0.2511 dict = 1.00 all = 0.2469 } ---------------------------- total value len = 9.192450 GB avg = 0.167 KB (by entry num) total key len = 1.519252 GB avg = 0.028 KB total ukey len = 1.053900 GB avg = 0.015 KB total ukey num = 0.068351524 Billion total entry num = 0.055159253 Billion write speed all = 0.175691974 MB/sec (with seq num) write speed all = 0.168454254 MB/sec (without seq num) 2024/11/24-01:51:20.903842 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287859: 942880 keys, 33134050 bytes, temperature: kWarm 2024/11/24-01:51:20.903919 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384280903879, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287859, "file_size": 33134050, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4381223929, "table_properties": {"data_size": 27076928, "index_size": 1793984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26400640, "raw_average_key_size": 28, "raw_value_size": 107817100, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 942880, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384277, "oldest_key_time": 0, "file_creation_time": 1732384277, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:20.938356 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst) /dev/shm/ToplingTemp/Topling-241780-pEhY5s 2024/11/24-01:51:21.880146 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst): first pass time = 0.94's, 142.519'MB/sec 2024/11/24-01:51:23.357925 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst): old prealloc_size = 147639500, real_size = 30716976 2024/11/24-01:51:23.358452 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926402 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.88 avg-zval = 30.99 2024/11/24-01:51:23.359213 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:24.164447 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287860.sst): second pass time = 1.48's, 73.312'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3499.239'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 276.979'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35686.063'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.312'MB/sec zip pipeline throughput = 111.698'MB/sec entries = 926402 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.88 avg-zval = 30.99 usrkeys = 926402 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.88 avg-zval = 30.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926402 bytes = 3126624 } seq expand size = 6484814 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4441 value = 3.7713 dict = 1.00 all = 3.8683 } Zip/UnZip{ index = 0.0957 value = 0.2652 dict = 1.00 all = 0.2585 } ---------------------------- total value len = 9.307214 GB avg = 0.166 KB (by entry num) total key len = 1.545191 GB avg = 0.028 KB total ukey len = 1.072428 GB avg = 0.015 KB total ukey num = 0.069277926 Billion total entry num = 0.056085655 Billion write speed all = 0.177990115 MB/sec (with seq num) write speed all = 0.170631236 MB/sec (without seq num) 2024/11/24-01:51:24.208518 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287860: 926402 keys, 34696590 bytes, temperature: kWarm 2024/11/24-01:51:24.208586 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384284208551, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287860, "file_size": 34696590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4277180558, "largest_seqno": 4381223600, "table_properties": {"data_size": 28711200, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939256, "raw_average_key_size": 28, "raw_value_size": 108278480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926402, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384280, "oldest_key_time": 0, "file_creation_time": 1732384280, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:24.244655 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst) /dev/shm/ToplingTemp/Topling-241780-BGXTLu 2024/11/24-01:51:25.139538 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst): first pass time = 0.89's, 149.990'MB/sec 2024/11/24-01:51:26.704812 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst): old prealloc_size = 147639500, real_size = 31800864 2024/11/24-01:51:26.705385 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936524 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0298 GB avg-val = 115.31 avg-zval = 31.80 2024/11/24-01:51:26.706167 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:27.570239 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287861.sst): second pass time = 1.56's, 69.051'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.171'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 269.851'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 22986.976'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.051'MB/sec zip pipeline throughput = 121.372'MB/sec entries = 936524 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.31 avg-zval = 31.80 usrkeys = 936524 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.31 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936524 bytes = 3160784 } seq expand size = 6555668 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5139 value = 3.6258 dict = 1.00 all = 3.7480 } Zip/UnZip{ index = 0.0951 value = 0.2758 dict = 1.00 all = 0.2668 } ---------------------------- total value len = 9.421764 GB avg = 0.165 KB (by entry num) total key len = 1.571414 GB avg = 0.028 KB total ukey len = 1.091158 GB avg = 0.016 KB total ukey num = 0.070214450 Billion total entry num = 0.057022179 Billion write speed all = 0.180288869 MB/sec (with seq num) write speed all = 0.172807528 MB/sec (without seq num) 2024/11/24-01:51:27.620462 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287861: 936524 keys, 35810056 bytes, temperature: kWarm 2024/11/24-01:51:27.620531 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384287620494, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287861, "file_size": 35810056, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4381224141, "table_properties": {"data_size": 29785040, "index_size": 1781504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26222672, "raw_average_key_size": 28, "raw_value_size": 107995164, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936524, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384284, "oldest_key_time": 0, "file_creation_time": 1732384284, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:27.655079 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst) /dev/shm/ToplingTemp/Topling-241780-4Q3Kwv 2024/11/24-01:51:28.557196 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst): first pass time = 0.90's, 148.787'MB/sec 2024/11/24-01:51:30.089752 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst): old prealloc_size = 147639500, real_size = 30562992 2024/11/24-01:51:30.090321 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927993 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.63 avg-zval = 30.77 2024/11/24-01:51:30.091097 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:30.910474 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287862.sst): second pass time = 1.53's, 70.662'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.094'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.843'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 266.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33664.516'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.662'MB/sec zip pipeline throughput = 122.235'MB/sec entries = 927993 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.63 avg-zval = 30.77 usrkeys = 927993 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.63 avg-zval = 30.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927993 bytes = 3131984 } seq expand size = 6495951 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4451 value = 3.7905 dict = 1.00 all = 3.8851 } Zip/UnZip{ index = 0.0957 value = 0.2638 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 9.536494 GB avg = 0.165 KB (by entry num) total key len = 1.597397 GB avg = 0.028 KB total ukey len = 1.109718 GB avg = 0.016 KB total ukey num = 0.071142443 Billion total entry num = 0.057950172 Billion write speed all = 0.182586582 MB/sec (with seq num) write speed all = 0.174983904 MB/sec (without seq num) 2024/11/24-01:51:30.989505 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287862: 927993 keys, 34546884 bytes, temperature: kWarm 2024/11/24-01:51:30.989584 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384290989546, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287862, "file_size": 34546884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4303172675, "largest_seqno": 4381223620, "table_properties": {"data_size": 28553888, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983804, "raw_average_key_size": 28, "raw_value_size": 108233982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927993, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384287, "oldest_key_time": 0, "file_creation_time": 1732384287, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:31.030451 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst) /dev/shm/ToplingTemp/Topling-241780-XQ1mOt 2024/11/24-01:51:31.850389 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst): first pass time = 0.82's, 163.699'MB/sec 2024/11/24-01:51:33.433020 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst): old prealloc_size = 147639500, real_size = 32760160 2024/11/24-01:51:33.433574 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924499 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.18 avg-zval = 33.28 2024/11/24-01:51:33.434328 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:34.275342 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287863.sst): second pass time = 1.58's, 68.492'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3314.737'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 293.098'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31216.515'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.492'MB/sec zip pipeline throughput = 118.324'MB/sec entries = 924499 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.28 usrkeys = 924499 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924499 bytes = 3120192 } seq expand size = 6471493 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5133 value = 3.5207 dict = 1.00 all = 3.6536 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 9.651298 GB avg = 0.164 KB (by entry num) total key len = 1.623283 GB avg = 0.028 KB total ukey len = 1.128208 GB avg = 0.016 KB total ukey num = 0.072066942 Billion total entry num = 0.058874671 Billion write speed all = 0.184883566 MB/sec (with seq num) write speed all = 0.177160026 MB/sec (without seq num) 2024/11/24-01:51:34.325020 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287863: 924499 keys, 36736240 bytes, temperature: kWarm 2024/11/24-01:51:34.325087 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384294325053, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287863, "file_size": 36736240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4381224279, "table_properties": {"data_size": 30770128, "index_size": 1758720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885972, "raw_average_key_size": 28, "raw_value_size": 108331762, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924499, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384291, "oldest_key_time": 0, "file_creation_time": 1732384291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:34.361783 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst) /dev/shm/ToplingTemp/Topling-241780-q9a0tv 2024/11/24-01:51:35.204032 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst): first pass time = 0.84's, 159.365'MB/sec 2024/11/24-01:51:36.809174 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst): old prealloc_size = 147639500, real_size = 32717504 2024/11/24-01:51:36.809679 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930404 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.26 avg-zval = 33.00 2024/11/24-01:51:36.811101 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:37.665655 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287864.sst): second pass time = 1.60's, 67.422'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.862'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3706.066'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 290.532'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34266.353'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.422'MB/sec zip pipeline throughput = 125.784'MB/sec entries = 930404 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 33.00 usrkeys = 930404 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930404 bytes = 3140128 } seq expand size = 6512828 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4444 value = 3.5230 dict = 1.00 all = 3.6562 } Zip/UnZip{ index = 0.0957 value = 0.2839 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 9.765977 GB avg = 0.163 KB (by entry num) total key len = 1.649335 GB avg = 0.028 KB total ukey len = 1.146816 GB avg = 0.016 KB total ukey num = 0.072997346 Billion total entry num = 0.059805075 Billion write speed all = 0.187180899 MB/sec (with seq num) write speed all = 0.179335739 MB/sec (without seq num) 2024/11/24-01:51:37.723011 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287864: 930404 keys, 36709530 bytes, temperature: kWarm 2024/11/24-01:51:37.723078 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384297723042, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287864, "file_size": 36709530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4381223639, "table_properties": {"data_size": 30703088, "index_size": 1781632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051312, "raw_average_key_size": 28, "raw_value_size": 108166544, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930404, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384294, "oldest_key_time": 0, "file_creation_time": 1732384294, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:37.766824 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst) /dev/shm/ToplingTemp/Topling-241780-l6YD2u 2024/11/24-01:51:38.639358 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst): first pass time = 0.87's, 153.833'MB/sec 2024/11/24-01:51:40.202238 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst): old prealloc_size = 147639500, real_size = 30926448 2024/11/24-01:51:40.202798 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925575 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.01 avg-zval = 31.25 2024/11/24-01:51:40.203571 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:41.025188 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287865.sst): second pass time = 1.56's, 69.331'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.414'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3297.401'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 278.745'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29706.267'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.331'MB/sec zip pipeline throughput = 119.662'MB/sec entries = 925575 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.01 avg-zval = 31.25 usrkeys = 925575 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.01 avg-zval = 31.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925575 bytes = 3123824 } seq expand size = 6479025 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4461 value = 3.7445 dict = 1.00 all = 3.8454 } Zip/UnZip{ index = 0.0957 value = 0.2671 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 9.880758 GB avg = 0.163 KB (by entry num) total key len = 1.675251 GB avg = 0.028 KB total ukey len = 1.165328 GB avg = 0.016 KB total ukey num = 0.073922921 Billion total entry num = 0.060730650 Billion write speed all = 0.189477517 MB/sec (with seq num) write speed all = 0.181511380 MB/sec (without seq num) 2024/11/24-01:51:41.070642 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287865: 925575 keys, 34903578 bytes, temperature: kWarm 2024/11/24-01:51:41.070708 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384301070672, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287865, "file_size": 34903578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4298365246, "largest_seqno": 4381223649, "table_properties": {"data_size": 28922784, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25916100, "raw_average_key_size": 28, "raw_value_size": 108301726, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925575, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384297, "oldest_key_time": 0, "file_creation_time": 1732384297, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:41.119731 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst) /dev/shm/ToplingTemp/Topling-241780-ps4Fns 2024/11/24-01:51:41.987514 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst): first pass time = 0.87's, 154.675'MB/sec 2024/11/24-01:51:43.528047 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst): old prealloc_size = 147639500, real_size = 32601920 2024/11/24-01:51:43.528608 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923337 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.36 avg-zval = 33.14 2024/11/24-01:51:43.529364 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:44.361254 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287866.sst): second pass time = 1.54's, 70.398'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.898'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.574'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 294.367'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32504.354'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.398'MB/sec zip pipeline throughput = 104.961'MB/sec entries = 923337 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.36 avg-zval = 33.14 usrkeys = 923337 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.36 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923337 bytes = 3116272 } seq expand size = 6463359 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4446 value = 3.5410 dict = 1.00 all = 3.6696 } Zip/UnZip{ index = 0.0957 value = 0.2824 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 9.995585 GB avg = 0.162 KB (by entry num) total key len = 1.701104 GB avg = 0.028 KB total ukey len = 1.183794 GB avg = 0.016 KB total ukey num = 0.074846258 Billion total entry num = 0.061653987 Billion write speed all = 0.191773698 MB/sec (with seq num) write speed all = 0.183686888 MB/sec (without seq num) 2024/11/24-01:51:44.407789 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287866: 923337 keys, 36575596 bytes, temperature: kWarm 2024/11/24-01:51:44.407855 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384304407821, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287866, "file_size": 36575596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4381223959, "table_properties": {"data_size": 30602864, "index_size": 1768064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25853436, "raw_average_key_size": 28, "raw_value_size": 108364318, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923337, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384301, "oldest_key_time": 0, "file_creation_time": 1732384301, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:44.449693 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst) /dev/shm/ToplingTemp/Topling-241780-bonakt 2024/11/24-01:51:45.328258 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst): first pass time = 0.88's, 152.777'MB/sec 2024/11/24-01:51:46.716515 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst): old prealloc_size = 147639500, real_size = 32956880 2024/11/24-01:51:46.717072 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929654 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.37 avg-zval = 33.30 2024/11/24-01:51:46.717825 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:47.569796 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287867.sst): second pass time = 1.39's, 77.973'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.302'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.736'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 292.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35104.650'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.973'MB/sec zip pipeline throughput = 77.973'MB/sec entries = 929654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 33.30 usrkeys = 929654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 33.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929654 bytes = 3137600 } seq expand size = 6507578 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5138 value = 3.4949 dict = 1.00 all = 3.6327 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 10.110280 GB avg = 0.162 KB (by entry num) total key len = 1.727134 GB avg = 0.028 KB total ukey len = 1.202388 GB avg = 0.016 KB total ukey num = 0.075775912 Billion total entry num = 0.062583641 Billion write speed all = 0.194070759 MB/sec (with seq num) write speed all = 0.185862443 MB/sec (without seq num) 2024/11/24-01:51:47.619282 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287867: 929654 keys, 36947388 bytes, temperature: kWarm 2024/11/24-01:51:47.619369 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384307619316, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287867, "file_size": 36947388, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4381224495, "table_properties": {"data_size": 30955840, "index_size": 1768448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030312, "raw_average_key_size": 28, "raw_value_size": 108187468, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384304, "oldest_key_time": 0, "file_creation_time": 1732384304, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:47.697106 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst) /dev/shm/ToplingTemp/Topling-241780-smvrDt 2024/11/24-01:51:48.708845 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst): first pass time = 1.01's, 132.668'MB/sec 2024/11/24-01:51:50.028902 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst): old prealloc_size = 147639500, real_size = 30638736 2024/11/24-01:51:50.029445 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927201 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.76 avg-zval = 30.88 2024/11/24-01:51:50.030195 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:50.983831 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287868.sst): second pass time = 1.32's, 82.058'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 36.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3438.283'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.23's, 149.265'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30432.917'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.058'MB/sec zip pipeline throughput = 82.058'MB/sec entries = 927201 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 30.88 usrkeys = 927201 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 30.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927201 bytes = 3129312 } seq expand size = 6490407 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4445 value = 3.7811 dict = 1.00 all = 3.8768 } Zip/UnZip{ index = 0.0957 value = 0.2645 dict = 1.00 all = 0.2579 } ---------------------------- total value len = 10.225027 GB avg = 0.161 KB (by entry num) total key len = 1.753096 GB avg = 0.028 KB total ukey len = 1.220932 GB avg = 0.016 KB total ukey num = 0.076703113 Billion total entry num = 0.063510842 Billion write speed all = 0.196366635 MB/sec (with seq num) write speed all = 0.188037176 MB/sec (without seq num) 2024/11/24-01:51:51.029542 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287868: 927201 keys, 34620788 bytes, temperature: kWarm 2024/11/24-01:51:51.029612 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384311029576, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287868, "file_size": 34620788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4305145959, "largest_seqno": 4381223680, "table_properties": {"data_size": 28631232, "index_size": 1775488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961628, "raw_average_key_size": 28, "raw_value_size": 108256138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927201, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384307, "oldest_key_time": 0, "file_creation_time": 1732384307, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:51.065464 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst) /dev/shm/ToplingTemp/Topling-241780-XEVApv 2024/11/24-01:51:52.059777 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst): first pass time = 0.99's, 134.991'MB/sec 2024/11/24-01:51:53.402410 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst): old prealloc_size = 147639500, real_size = 32334096 2024/11/24-01:51:53.402956 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921980 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.58 avg-zval = 32.91 2024/11/24-01:51:53.404374 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:54.367314 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287869.sst): second pass time = 1.34's, 80.844'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 37.626'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.945'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.17's, 208.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 27237.013'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.844'MB/sec zip pipeline throughput = 80.844'MB/sec entries = 921980 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.58 avg-zval = 32.91 usrkeys = 921980 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.58 avg-zval = 32.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921980 bytes = 3111696 } seq expand size = 6453860 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4440 value = 3.5732 dict = 1.00 all = 3.6970 } Zip/UnZip{ index = 0.0957 value = 0.2799 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 10.339883 GB avg = 0.160 KB (by entry num) total key len = 1.778912 GB avg = 0.028 KB total ukey len = 1.239371 GB avg = 0.016 KB total ukey num = 0.077625093 Billion total entry num = 0.064432822 Billion write speed all = 0.198661756 MB/sec (with seq num) write speed all = 0.190211848 MB/sec (without seq num) 2024/11/24-01:51:54.413560 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287869: 921980 keys, 36304352 bytes, temperature: kWarm 2024/11/24-01:51:54.413629 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384314413596, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287869, "file_size": 36304352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4381224342, "table_properties": {"data_size": 30337856, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25815440, "raw_average_key_size": 28, "raw_value_size": 108402412, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921980, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384311, "oldest_key_time": 0, "file_creation_time": 1732384311, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:54.453615 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst) /dev/shm/ToplingTemp/Topling-241780-H6DhFr 2024/11/24-01:51:55.470554 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst): first pass time = 1.02's, 131.986'MB/sec 2024/11/24-01:51:56.814585 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst): old prealloc_size = 147639500, real_size = 32954560 2024/11/24-01:51:56.815152 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.35 avg-zval = 33.29 2024/11/24-01:51:56.816550 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:57.677410 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287870.sst): second pass time = 1.34's, 80.540'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.133'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 290.187'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30940.023'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.540'MB/sec zip pipeline throughput = 80.540'MB/sec entries = 929794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 33.29 usrkeys = 929794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929794 bytes = 3138064 } seq expand size = 6508558 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5138 value = 3.4951 dict = 1.00 all = 3.6329 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 10.454575 GB avg = 0.160 KB (by entry num) total key len = 1.804946 GB avg = 0.028 KB total ukey len = 1.257967 GB avg = 0.016 KB total ukey num = 0.078554887 Billion total entry num = 0.065362616 Billion write speed all = 0.200957759 MB/sec (with seq num) write speed all = 0.192386380 MB/sec (without seq num) 2024/11/24-01:51:57.722308 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287870: 929794 keys, 36945188 bytes, temperature: kWarm 2024/11/24-01:51:57.722376 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384317722342, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287870, "file_size": 36945188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4381224238, "table_properties": {"data_size": 30953200, "index_size": 1768704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034232, "raw_average_key_size": 28, "raw_value_size": 108183552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384314, "oldest_key_time": 0, "file_creation_time": 1732384314, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:57.757251 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst) /dev/shm/ToplingTemp/Topling-241780-VQOS2t 2024/11/24-01:51:58.618198 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst): first pass time = 0.86's, 155.902'MB/sec 2024/11/24-01:51:59.906814 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst): old prealloc_size = 147639500, real_size = 30584160 2024/11/24-01:51:59.907396 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.56 avg-zval = 30.76 2024/11/24-01:51:59.909321 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:52:00.737108 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287871.sst): second pass time = 1.29's, 84.074'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.636'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.880'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 266.552'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34597.680'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.074'MB/sec zip pipeline throughput = 84.074'MB/sec entries = 928462 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.56 avg-zval = 30.76 usrkeys = 928462 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.56 avg-zval = 30.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928462 bytes = 3133568 } seq expand size = 6499234 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3756 value = 3.7890 dict = 1.00 all = 3.8826 } Zip/UnZip{ index = 0.0964 value = 0.2639 dict = 1.00 all = 0.2576 } ---------------------------- total value len = 10.569295 GB avg = 0.159 KB (by entry num) total key len = 1.830943 GB avg = 0.028 KB total ukey len = 1.276536 GB avg = 0.016 KB total ukey num = 0.079483349 Billion total entry num = 0.066291078 Billion write speed all = 0.203254194 MB/sec (with seq num) write speed all = 0.194561497 MB/sec (without seq num) 2024/11/24-01:52:00.780335 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287871: 928462 keys, 34569330 bytes, temperature: kWarm 2024/11/24-01:52:00.780406 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384320780371, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287871, "file_size": 34569330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310322247, "largest_seqno": 4381223710, "table_properties": {"data_size": 28562192, "index_size": 1789696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996936, "raw_average_key_size": 28, "raw_value_size": 108220812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384317, "oldest_key_time": 0, "file_creation_time": 1732384317, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:52:00.819750 139957609621056 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst) /dev/shm/ToplingTemp/Topling-241780-08En4s 2024/11/24-01:52:00.982674 139957609621056 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst): first pass time = 0.16's, 242.472'MB/sec 2024/11/24-01:52:01.367185 139957609621056 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst): old prealloc_size = 147639500, real_size = 11161040 2024/11/24-01:52:01.367395 139957609621056 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 271855 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0319 GB zip-val = 0.0106 GB avg-val = 117.27 avg-zval = 38.88 2024/11/24-01:52:01.367658 139957609621056 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.039 GB 2024/11/24-01:52:01.610797 139957609621056 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00/287872.sst): second pass time = 0.38's, 83.075'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 50.652'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2788.632'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.331'MB/sec write SST data time = 0.04's, 324.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.325'MB, throughput = 10795.550'MB/sec zip my value time = 0.38's, unzip length = 0.032'GB zip my value throughput = 83.075'MB/sec zip pipeline throughput = 83.075'MB/sec entries = 271855 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.27 avg-zval = 38.88 usrkeys = 271855 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.27 avg-zval = 38.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 271855 bytes = 917520 } seq expand size = 1902985 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0054 GB value = 0.0319 GB dict = 0.33 MB all = 0.0395 GB } __ZipSize{ index = 0.0005 GB value = 0.0106 GB dict = 0.33 MB all = 0.0123 GB } UnZip/Zip{ index = 10.4201 value = 3.0159 dict = 1.00 all = 3.2009 } Zip/UnZip{ index = 0.0960 value = 0.3316 dict = 1.00 all = 0.3124 } ---------------------------- total value len = 10.603079 GB avg = 0.159 KB (by entry num) total key len = 1.838555 GB avg = 0.028 KB total ukey len = 1.281973 GB avg = 0.016 KB total ukey num = 0.079755204 Billion total entry num = 0.066562933 Billion write speed all = 0.203929801 MB/sec (with seq num) write speed all = 0.195201581 MB/sec (without seq num) 2024/11/24-01:52:01.631259 139957609621056 [compaction_job.cc:1948] [default] [JOB 39641] Generated table #287872: 271855 keys, 12338243 bytes, temperature: kWarm 2024/11/24-01:52:01.631327 139957609621056 EVENT_LOG_v1 {"time_micros": 1732384321631292, "cf_name": "default", "job": 39641, "event": "table_file_creation", "file_number": 287872, "file_size": 12338243, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4381224539, "table_properties": {"data_size": 10571072, "index_size": 521792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7611940, "raw_average_key_size": 28, "raw_value_size": 31881038, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 271855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384320, "oldest_key_time": 0, "file_creation_time": 1732384320, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:52:01.654146 139957609621056 [compaction_job.cc:703] job-39641: subcompact[0], size: 0.548361 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39641/att-00 [287857,287858,287859,287860,287861,287862,287863,287864,287865,287866,287867,287868,287869,287870,287871,287872] 2024/11/24-01:52:01.748797 139957609621056 (Original Log Time 2024/11/24-01:51:10.590398) EVENT_LOG_v1 {"time_micros": 1732384270590358, "job": 39641, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [287814], "files_L3": [287854, 287762, 287763, 287764, 287765, 287766, 287767, 287768, 287769, 287770, 287771, 287779, 287780, 287781, 287782, 287835], "score": 0.542744, "input_data_size": 570720721, "oldest_snapshot_seqno": 4381418007} 2024/11/24-01:52:01.748801 139957609621056 (Original Log Time 2024/11/24-01:52:01.747405) [compaction_job.cc:2022] [default] [JOB 39641] Compacted 1@2 + 16@3 files to L3 => 548361487 bytes 2024/11/24-01:52:01.748803 139957609621056 (Original Log Time 2024/11/24-01:52:01.748715) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.7 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 517.2 +0.0 blob) out(523.0 +0.0 blob), read-write-amplify(39.4) write-amplify(19.3) OK, records in: 14300297, records dropped: 95528 output_compression: Snappy 2024/11/24-01:52:01.748810 139957609621056 (Original Log Time 2024/11/24-01:52:01.748776) EVENT_LOG_v1 {"time_micros": 1732384321748749, "cf": "default", "job": 39641, "event": "compaction_finished", "compaction_time_micros": 51063758, "compaction_time_cpu_micros": 21671236, "output_level": 3, "num_output_files": 16, "total_output_size": 548361487, "num_input_records": 14300297, "num_output_records": 14204769, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 396888147, "file_range_sync_nanos": 0, "file_fsync_nanos": 11507, "file_prepare_write_nanos": 48111, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/24-01:52:01.878820 139957609621056 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732384270,"executesMs":51296,"compactionJobId":39641,"attempt":0,"compactionInputRawBytes":2066324750,"compactionInputZipBytes":570720721,"compactionOutputRawBytes":0,"compactionOutputZipBytes":548361487,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9c2a1a0a412e79bf0e2176ce8f2f6135-ef516aba1a47f5e2-00"}}, response = 2024/11/24-01:52:01.878934 139957609621056 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39641/att-00: olev 3, work 51.297 s, result 7.786 ms, install 1.423 ms, input{raw 1.924 GiB zip 544.282 MiB}