2024/11/23-01:23:39.947747 140025028785728 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00: fork to child time = 0.009725 sec 2024/11/23-01:23:39.948660 140025028785728 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00 2024/11/23-01:23:39.956549 140025028785728 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00 2024/11/23-01:23:39.957674 140025028785728 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-01:23:39.958050 140025028785728 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-01:23:40.010245 140025028785728 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16291/att-00/MANIFEST-206347 2024/11/23-01:23:40.039543 140025028785728 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16291/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 238490, last_sequence is 3703711305, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 238338 2024/11/23-01:23:40.039551 140025028785728 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-01:23:40.039639 140025028785728 [version_set.cc:5721] Creating manifest 238490 2024/11/23-01:23:40.075103 140025028785728 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-01:23:40.097586 140025028785728 [compaction_job.cc:2437] [default] [JOB 16291] Compacting 1@2 + 10@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-01:23:40.097605 140025028785728 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 12199 Base level 2, inputs: [238488(23MB)], [238411(35MB) 238412(36MB) 238413(35MB) 238414(37MB) 238415(37MB) 238416(35MB) 238417(35MB) 238418(36MB) 238419(35MB) 238420(34MB)] 2024/11/23-01:23:40.103168 140025028785728 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:23:40.103199 140025028785728 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:23:40.103223 140025028785728 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:23:40.103229 140025028785728 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:23:40.103237 140025028785728 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:23:40.607660 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst) /dev/shm/ToplingTemp/Topling-365879-x8ZKTd 2024/11/23-01:23:41.222495 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst): first pass time = 0.61's, 218.314'MB/sec 2024/11/23-01:23:43.387456 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst): old prealloc_size = 147639500, real_size = 33277584 2024/11/23-01:23:43.388068 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932220 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.98 avg-zval = 33.55 2024/11/23-01:23:43.388908 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:23:44.894525 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238491.sst): second pass time = 2.16's, 49.965'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.51's, 24.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.873'MB/sec (index lex order gen) rebuild zvType time = 1.27's, 0.184'MB/sec write SST data time = 0.24's, 155.405'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32400.385'MB/sec zip my value time = 2.16's, unzip length = 0.108'GB zip my value throughput = 49.965'MB/sec zip pipeline throughput = 107.780'MB/sec entries = 932220 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 33.55 usrkeys = 932220 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932220 bytes = 3379312 } seq expand size = 6525540 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5158 value = 3.4573 dict = 1.00 all = 3.5784 } Zip/UnZip{ index = 0.0951 value = 0.2892 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 0.114641 GB avg = 0.123 KB (by entry num) total key len = 0.026102 GB avg = 0.028 KB total ukey len = 0.018644 GB avg = 0.020 KB total ukey num = 0.000932220 Billion total entry num = 0.000932220 Billion write speed all = 0.002366157 MB/sec (with seq num) write speed all = 0.002240778 MB/sec (without seq num) 2024/11/23-01:23:44.932382 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238491: 932220 keys, 37507706 bytes, temperature: kWarm 2024/11/23-01:23:44.932492 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296224932425, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238491, "file_size": 37507706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3703685647, "table_properties": {"data_size": 31271360, "index_size": 1772992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102160, "raw_average_key_size": 28, "raw_value_size": 108115576, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932220, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296220, "oldest_key_time": 0, "file_creation_time": 1732296220, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238491, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:23:45.005354 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst) /dev/shm/ToplingTemp/Topling-365879-qHCRyh 2024/11/23-01:23:46.618961 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst): first pass time = 1.61's, 83.181'MB/sec 2024/11/23-01:23:48.263512 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst): old prealloc_size = 147639500, real_size = 34520128 2024/11/23-01:23:48.264202 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929326 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.42 avg-zval = 34.98 2024/11/23-01:23:48.265043 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:23:49.931810 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238492.sst): second pass time = 1.64's, 65.826'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.67's, 23.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2653.886'MB/sec (index lex order gen) rebuild zvType time = 1.30's, 0.178'MB/sec write SST data time = 0.36's, 106.351'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36702.594'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.826'MB/sec zip pipeline throughput = 65.826'MB/sec entries = 929326 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 34.98 usrkeys = 929326 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.42 avg-zval = 34.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929326 bytes = 3368816 } seq expand size = 6505282 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4439 value = 3.3283 dict = 1.00 all = 3.4644 } Zip/UnZip{ index = 0.0957 value = 0.3005 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 0.229343 GB avg = 0.123 KB (by entry num) total key len = 0.052123 GB avg = 0.028 KB total ukey len = 0.037231 GB avg = 0.020 KB total ukey num = 0.001861546 Billion total entry num = 0.001861546 Billion write speed all = 0.004731573 MB/sec (with seq num) write speed all = 0.004481225 MB/sec (without seq num) 2024/11/23-01:23:49.968453 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238492: 929326 keys, 38741812 bytes, temperature: kWarm 2024/11/23-01:23:49.968536 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296229968492, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238492, "file_size": 38741812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3703685798, "table_properties": {"data_size": 32507952, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021128, "raw_average_key_size": 28, "raw_value_size": 108196604, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296224, "oldest_key_time": 0, "file_creation_time": 1732296224, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238492, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:23:50.049454 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst) /dev/shm/ToplingTemp/Topling-365879-s5R5pe 2024/11/23-01:23:51.686399 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst): first pass time = 1.64's, 81.996'MB/sec 2024/11/23-01:23:53.436738 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst): old prealloc_size = 147639500, real_size = 33619984 2024/11/23-01:23:53.437446 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932718 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.90 avg-zval = 33.87 2024/11/23-01:23:53.438284 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:23:54.389060 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238493.sst): second pass time = 1.75's, 61.787'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.747'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2607.241'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.15's, 257.326'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38586.108'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.787'MB/sec zip pipeline throughput = 73.589'MB/sec entries = 932718 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.90 avg-zval = 33.87 usrkeys = 932718 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.90 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932718 bytes = 3381120 } seq expand size = 6529026 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.3783 value = 3.4221 dict = 1.00 all = 3.5459 } Zip/UnZip{ index = 0.0964 value = 0.2922 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 0.343974 GB avg = 0.123 KB (by entry num) total key len = 0.078239 GB avg = 0.028 KB total ukey len = 0.055885 GB avg = 0.020 KB total ukey num = 0.002794264 Billion total entry num = 0.002794264 Billion write speed all = 0.007097056 MB/sec (with seq num) write speed all = 0.006721302 MB/sec (without seq num) 2024/11/23-01:23:54.422115 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238493: 932718 keys, 37851486 bytes, temperature: kWarm 2024/11/23-01:23:54.422184 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296234422148, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238493, "file_size": 37851486, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3703685737, "table_properties": {"data_size": 31589184, "index_size": 1797440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26116104, "raw_average_key_size": 28, "raw_value_size": 108101640, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296230, "oldest_key_time": 0, "file_creation_time": 1732296230, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238493, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:23:54.461089 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst) /dev/shm/ToplingTemp/Topling-365879-ZSLMzg 2024/11/23-01:23:55.921461 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst): first pass time = 1.46's, 91.909'MB/sec 2024/11/23-01:23:58.265216 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst): old prealloc_size = 147639500, real_size = 35107456 2024/11/23-01:23:58.265799 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929704 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.37 avg-zval = 35.61 2024/11/23-01:23:58.266576 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:23:59.434586 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238494.sst): second pass time = 2.34's, 46.173'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 33.635'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.900'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.244'MB/sec write SST data time = 0.22's, 180.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30961.348'MB/sec zip my value time = 2.34's, unzip length = 0.108'GB zip my value throughput = 46.173'MB/sec zip pipeline throughput = 151.612'MB/sec entries = 929704 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 35.61 usrkeys = 929704 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 35.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929704 bytes = 3370192 } seq expand size = 6507928 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5128 value = 3.2679 dict = 1.00 all = 3.4126 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2930 } ---------------------------- total value len = 0.458668 GB avg = 0.123 KB (by entry num) total key len = 0.104271 GB avg = 0.028 KB total ukey len = 0.074479 GB avg = 0.020 KB total ukey num = 0.003723968 Billion total entry num = 0.003723968 Billion write speed all = 0.009461738 MB/sec (with seq num) write speed all = 0.008961005 MB/sec (without seq num) 2024/11/23-01:23:59.473131 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238494: 929704 keys, 39330060 bytes, temperature: kWarm 2024/11/23-01:23:59.473244 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296239473200, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238494, "file_size": 39330060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3703685575, "table_properties": {"data_size": 33106160, "index_size": 1768704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26031712, "raw_average_key_size": 28, "raw_value_size": 108186060, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296234, "oldest_key_time": 0, "file_creation_time": 1732296234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238494, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:23:59.582012 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst) /dev/shm/ToplingTemp/Topling-365879-L4QAEd 2024/11/23-01:24:00.679415 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst): first pass time = 1.10's, 122.310'MB/sec 2024/11/23-01:24:02.373531 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst): old prealloc_size = 147639500, real_size = 35562432 2024/11/23-01:24:02.374124 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928478 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.56 avg-zval = 36.14 2024/11/23-01:24:02.374950 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:03.354820 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238495.sst): second pass time = 1.69's, 63.910'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 40.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3111.239'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.16's, 241.331'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34198.581'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.910'MB/sec zip pipeline throughput = 63.910'MB/sec entries = 928478 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.56 avg-zval = 36.14 usrkeys = 928478 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.96 avg-zval = 36.14 TagRS{ kinds = RS_Key0_TagN bytes = 145160 } TagArray{ size = 848717 bytes = 3076608 } seq expand size = 5941019 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4438 value = 3.2254 dict = 1.00 all = 3.3858 } Zip/UnZip{ index = 0.0958 value = 0.3100 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 0.572829 GB avg = 0.123 KB (by entry num) total key len = 0.130268 GB avg = 0.028 KB total ukey len = 0.093049 GB avg = 0.020 KB total ukey num = 0.004652446 Billion total entry num = 0.004652446 Billion write speed all = 0.011816715 MB/sec (with seq num) write speed all = 0.011191178 MB/sec (without seq num) 2024/11/23-01:24:03.389418 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238495: 928478 keys, 39641124 bytes, temperature: kWarm 2024/11/23-01:24:03.389482 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296243389450, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238495, "file_size": 39641124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3703685264, "table_properties": {"data_size": 33552112, "index_size": 1778048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997384, "raw_average_key_size": 28, "raw_value_size": 108220412, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928478, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296239, "oldest_key_time": 0, "file_creation_time": 1732296239, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238495, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:03.434407 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst) /dev/shm/ToplingTemp/Topling-365879-474gOd 2024/11/23-01:24:04.471932 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst): first pass time = 1.04's, 129.369'MB/sec 2024/11/23-01:24:06.111813 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst): old prealloc_size = 147639500, real_size = 36400912 2024/11/23-01:24:06.112320 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926901 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.80 avg-zval = 37.13 2024/11/23-01:24:06.112366 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:06.803948 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238496.sst): second pass time = 1.64's, 66.073'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.832'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3634.013'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.17's, 215.924'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31156.972'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 66.073'MB/sec zip pipeline throughput = 66.073'MB/sec entries = 926901 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.80 avg-zval = 37.13 usrkeys = 926901 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.80 avg-zval = 37.13 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5834 value = 3.1456 dict = 1.00 all = 3.6023 } Zip/UnZip{ index = 0.0945 value = 0.3179 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.681094 GB avg = 0.122 KB (by entry num) total key len = 0.156222 GB avg = 0.028 KB total ukey len = 0.111587 GB avg = 0.020 KB total ukey num = 0.005579347 Billion total entry num = 0.005579347 Billion write speed all = 0.014071651 MB/sec (with seq num) write speed all = 0.013321534 MB/sec (without seq num) 2024/11/23-01:24:06.840285 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238496: 926901 keys, 37259058 bytes, temperature: kWarm 2024/11/23-01:24:06.840350 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296246840318, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238496, "file_size": 37259058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34417408, "index_size": 1751616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953228, "raw_average_key_size": 28, "raw_value_size": 108264630, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926901, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296243, "oldest_key_time": 0, "file_creation_time": 1732296243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238496, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:06.874984 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst) /dev/shm/ToplingTemp/Topling-365879-APfCJd 2024/11/23-01:24:07.737400 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst): first pass time = 0.86's, 155.638'MB/sec 2024/11/23-01:24:09.297129 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst): old prealloc_size = 147639500, real_size = 36434896 2024/11/23-01:24:09.297708 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927074 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.16 2024/11/23-01:24:09.297756 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:09.946719 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238497.sst): second pass time = 1.56's, 69.443'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.658'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.445'MB/sec write SST data time = 0.13's, 291.859'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29677.967'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.443'MB/sec zip pipeline throughput = 69.443'MB/sec entries = 927074 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.16 usrkeys = 927074 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5823 value = 3.1424 dict = 1.00 all = 3.5990 } Zip/UnZip{ index = 0.0945 value = 0.3182 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 0.789353 GB avg = 0.121 KB (by entry num) total key len = 0.182180 GB avg = 0.028 KB total ukey len = 0.130128 GB avg = 0.020 KB total ukey num = 0.006506421 Billion total entry num = 0.006506421 Billion write speed all = 0.016326410 MB/sec (with seq num) write speed all = 0.015451697 MB/sec (without seq num) 2024/11/23-01:24:09.980686 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238497: 927074 keys, 37292888 bytes, temperature: kWarm 2024/11/23-01:24:09.980749 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296249980717, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238497, "file_size": 37292888, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34450816, "index_size": 1752128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958072, "raw_average_key_size": 28, "raw_value_size": 108259712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927074, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296246, "oldest_key_time": 0, "file_creation_time": 1732296246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238497, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:10.018277 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst) /dev/shm/ToplingTemp/Topling-365879-wrZDYf 2024/11/23-01:24:10.965021 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst): first pass time = 0.95's, 141.774'MB/sec 2024/11/23-01:24:12.424524 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst): old prealloc_size = 147639500, real_size = 34716320 2024/11/23-01:24:12.425059 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928518 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.55 avg-zval = 35.25 2024/11/23-01:24:12.425706 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:13.290822 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238498.sst): second pass time = 1.46's, 74.181'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.686'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 292.826'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32677.402'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.181'MB/sec zip pipeline throughput = 74.181'MB/sec entries = 928518 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.55 avg-zval = 35.25 usrkeys = 928518 avg-key = 20.00 avg-zkey = 1.89 avg-val = 121.71 avg-zval = 35.25 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 684483 bytes = 2481264 } seq expand size = 4791381 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5817 value = 3.3065 dict = 1.00 all = 3.5136 } Zip/UnZip{ index = 0.0945 value = 0.3024 dict = 1.00 all = 0.2846 } ---------------------------- total value len = 0.902364 GB avg = 0.121 KB (by entry num) total key len = 0.208178 GB avg = 0.028 KB total ukey len = 0.148699 GB avg = 0.020 KB total ukey num = 0.007434939 Billion total entry num = 0.007434939 Billion write speed all = 0.018661380 MB/sec (with seq num) write speed all = 0.017661895 MB/sec (without seq num) 2024/11/23-01:24:13.325820 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238498: 928518 keys, 38199740 bytes, temperature: kWarm 2024/11/23-01:24:13.325886 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296253325854, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238498, "file_size": 38199740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3703685909, "table_properties": {"data_size": 32729040, "index_size": 1754944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998504, "raw_average_key_size": 28, "raw_value_size": 108219248, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928518, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296250, "oldest_key_time": 0, "file_creation_time": 1732296250, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238498, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:13.361163 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst) /dev/shm/ToplingTemp/Topling-365879-W0I9Ag 2024/11/23-01:24:14.389345 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst): first pass time = 1.03's, 130.545'MB/sec 2024/11/23-01:24:15.715378 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst): old prealloc_size = 147639500, real_size = 33121584 2024/11/23-01:24:15.715913 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926279 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.90 avg-zval = 33.60 2024/11/23-01:24:15.716651 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:16.651905 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238499.sst): second pass time = 1.33's, 81.710'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.618'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.473'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 278.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25279.594'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.710'MB/sec zip pipeline throughput = 81.710'MB/sec entries = 926279 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 33.60 usrkeys = 926279 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 33.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926279 bytes = 3126208 } seq expand size = 6483953 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5121 value = 3.4787 dict = 1.00 all = 3.6175 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.017130 GB avg = 0.122 KB (by entry num) total key len = 0.234114 GB avg = 0.028 KB total ukey len = 0.167224 GB avg = 0.020 KB total ukey num = 0.008361218 Billion total entry num = 0.008361218 Billion write speed all = 0.021024524 MB/sec (with seq num) write speed all = 0.019900582 MB/sec (without seq num) 2024/11/23-01:24:16.688139 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238499: 926279 keys, 37102470 bytes, temperature: kWarm 2024/11/23-01:24:16.688205 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296256688171, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238499, "file_size": 37102470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3573506306, "largest_seqno": 3703685925, "table_properties": {"data_size": 31127520, "index_size": 1762304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935812, "raw_average_key_size": 28, "raw_value_size": 108282046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926279, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296253, "oldest_key_time": 0, "file_creation_time": 1732296253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238499, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:16.729105 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst) /dev/shm/ToplingTemp/Topling-365879-XFmE5g 2024/11/23-01:24:17.756218 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst): first pass time = 1.03's, 130.680'MB/sec 2024/11/23-01:24:19.116845 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst): old prealloc_size = 147639500, real_size = 33153152 2024/11/23-01:24:19.117420 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928374 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.57 avg-zval = 33.55 2024/11/23-01:24:19.118207 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:24:20.067570 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238500.sst): second pass time = 1.36's, 79.579'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.296'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 294.949'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30478.457'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.579'MB/sec zip pipeline throughput = 79.579'MB/sec entries = 928374 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.57 avg-zval = 33.55 usrkeys = 928374 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.57 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928374 bytes = 3365376 } seq expand size = 6498618 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4419 value = 3.4751 dict = 1.00 all = 3.5914 } Zip/UnZip{ index = 0.0958 value = 0.2878 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 1.131852 GB avg = 0.122 KB (by entry num) total key len = 0.260109 GB avg = 0.028 KB total ukey len = 0.185792 GB avg = 0.020 KB total ukey num = 0.009289592 Billion total entry num = 0.009289592 Billion write speed all = 0.023387623 MB/sec (with seq num) write speed all = 0.022138959 MB/sec (without seq num) 2024/11/23-01:24:20.101358 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238500: 928374 keys, 37371662 bytes, temperature: kWarm 2024/11/23-01:24:20.101423 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296260101391, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238500, "file_size": 37371662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3703685941, "table_properties": {"data_size": 31142704, "index_size": 1778176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994472, "raw_average_key_size": 28, "raw_value_size": 108223260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928374, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296256, "oldest_key_time": 0, "file_creation_time": 1732296256, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238500, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:20.137963 140025028785728 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst) /dev/shm/ToplingTemp/Topling-365879-OjLHJg 2024/11/23-01:24:20.225054 140025028785728 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst): first pass time = 0.09's, 235.413'MB/sec 2024/11/23-01:24:20.407064 140025028785728 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst): old prealloc_size = 147639500, real_size = 5575504 2024/11/23-01:24:20.407198 140025028785728 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 144314 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0165 GB zip-val = 0.0053 GB avg-val = 114.00 avg-zval = 36.46 2024/11/23-01:24:20.407370 140025028785728 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/23-01:24:20.542187 140025028785728 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00/238501.sst): second pass time = 0.18's, 90.707'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 45.630'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2433.426'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.314'MB/sec write SST data time = 0.02's, 305.590'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.173'MB, throughput = 5454.098'MB/sec zip my value time = 0.18's, unzip length = 0.016'GB zip my value throughput = 90.707'MB/sec zip pipeline throughput = 90.707'MB/sec entries = 144314 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 36.46 usrkeys = 144314 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 36.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 144314 bytes = 450992 } seq expand size = 1010198 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0165 GB dict = 0.17 MB all = 0.0205 GB } __ZipSize{ index = 0.0003 GB value = 0.0053 GB dict = 0.17 MB all = 0.0062 GB } UnZip/Zip{ index = 10.3961 value = 3.1267 dict = 1.00 all = 3.3242 } Zip/UnZip{ index = 0.0962 value = 0.3198 dict = 1.00 all = 0.3008 } ---------------------------- total value len = 1.149314 GB avg = 0.122 KB (by entry num) total key len = 0.264149 GB avg = 0.028 KB total ukey len = 0.188678 GB avg = 0.020 KB total ukey num = 0.009433906 Billion total entry num = 0.009433906 Billion write speed all = 0.023748722 MB/sec (with seq num) write speed all = 0.022480670 MB/sec (without seq num) 2024/11/23-01:24:20.553078 140025028785728 [compaction_job.cc:1948] [default] [JOB 16291] Generated table #238501: 144314 keys, 6164727 bytes, temperature: kWarm 2024/11/23-01:24:20.553137 140025028785728 EVENT_LOG_v1 {"time_micros": 1732296260553105, "cf_name": "default", "job": 16291, "event": "table_file_creation", "file_number": 238501, "file_size": 6164727, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3668155657, "largest_seqno": 3693997575, "table_properties": {"data_size": 5261632, "index_size": 277632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4040792, "raw_average_key_size": 28, "raw_value_size": 16451796, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 144314, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296260, "oldest_key_time": 0, "file_creation_time": 1732296260, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238501, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:24:20.569928 140025028785728 [compaction_job.cc:703] job-16291: subcompact[0], size: 0.386463 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16291/att-00 [238491,238492,238493,238494,238495,238496,238497,238498,238499,238500,238501] 2024/11/23-01:24:20.641023 140025028785728 (Original Log Time 2024/11/23-01:23:40.097669) EVENT_LOG_v1 {"time_micros": 1732296220097622, "job": 16291, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [238488], "files_L3": [238411, 238412, 238413, 238414, 238415, 238416, 238417, 238418, 238419, 238420], "score": 0.997547, "input_data_size": 403992634, "oldest_snapshot_seqno": 3703711305} 2024/11/23-01:24:20.641027 140025028785728 (Original Log Time 2024/11/23-01:24:20.640387) [compaction_job.cc:2022] [default] [JOB 16291] Compacted 1@2 + 10@3 files to L3 => 386462733 bytes 2024/11/23-01:24:20.641028 140025028785728 (Original Log Time 2024/11/23-01:24:20.640959) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 10.0 rd, 9.5 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(23.9, 361.3 +0.0 blob) out(368.6 +0.0 blob), read-write-amplify(31.5) write-amplify(15.4) OK, records in: 9498061, records dropped: 64155 output_compression: Snappy 2024/11/23-01:24:20.641035 140025028785728 (Original Log Time 2024/11/23-01:24:20.641002) EVENT_LOG_v1 {"time_micros": 1732296260640982, "cf": "default", "job": 16291, "event": "compaction_finished", "compaction_time_micros": 40472267, "compaction_time_cpu_micros": 17518836, "output_level": 3, "num_output_files": 11, "total_output_size": 386462733, "num_input_records": 9498061, "num_output_records": 9433906, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 283925139, "file_range_sync_nanos": 0, "file_fsync_nanos": 11031, "file_prepare_write_nanos": 27647, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-01:24:20.762131 140025028785728 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732296219,"executesMs":40686,"compactionJobId":16291,"attempt":0,"compactionInputRawBytes":1371780362,"compactionInputZipBytes":403992634,"compactionOutputRawBytes":0,"compactionOutputZipBytes":386462733,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a60a7426ff928e7fb8226465d0d2ba0f-93adb1fa51d190b9-00"}}, response = 2024/11/23-01:24:20.762242 140025028785728 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16291/att-00: olev 3, work 40.686 s, result 6.354 ms, install 0.670 ms, input{raw 1.278 GiB zip 385.277 MiB}