2024/11/23-12:23:21.084188 139957341054528 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00: fork to child time = 0.007586 sec 2024/11/23-12:23:21.084787 139957341054528 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00 2024/11/23-12:23:21.091802 139957341054528 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00 2024/11/23-12:23:21.092710 139957341054528 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-12:23:21.092954 139957341054528 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-12:23:21.145360 139957341054528 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25938/att-00/MANIFEST-206347 2024/11/23-12:23:21.186203 139957341054528 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25938/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 259368, last_sequence is 3972505444, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 259145 2024/11/23-12:23:21.186213 139957341054528 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-12:23:21.186335 139957341054528 [version_set.cc:5721] Creating manifest 259368 2024/11/23-12:23:21.218823 139957341054528 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-12:23:21.239668 139957341054528 [compaction_job.cc:2437] [default] [JOB 25938] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-12:23:21.239682 139957341054528 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 19442 Base level 2, inputs: [259294(27MB)], [259366(8442KB) 259254(35MB) 259255(34MB) 259256(35MB) 259257(34MB) 259258(34MB) 259259(36MB) 259260(34MB) 259261(36MB) 259262(32MB) 259263(35MB) 259264(35MB) 259265(13MB) 259234(34MB) 259235(37MB) 259346(35MB)] 2024/11/23-12:23:21.248060 139957341054528 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:23:21.248089 139957341054528 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:23:21.248108 139957341054528 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-12:23:21.248114 139957341054528 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-12:23:21.248124 139957341054528 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-12:23:21.697114 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst) /dev/shm/ToplingTemp/Topling-47532-H2m2Ln 2024/11/23-12:23:22.607931 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst): first pass time = 0.91's, 147.367'MB/sec 2024/11/23-12:23:23.932607 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst): old prealloc_size = 147639500, real_size = 32434432 2024/11/23-12:23:23.933182 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934261 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.66 avg-zval = 32.54 2024/11/23-12:23:23.934035 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:24.776939 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259369.sst): second pass time = 1.32's, 81.631'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.572'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.798'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 282.442'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32092.078'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.631'MB/sec zip pipeline throughput = 81.631'MB/sec entries = 934261 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.66 avg-zval = 32.54 usrkeys = 934261 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.66 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934261 bytes = 3503488 } seq expand size = 6539827 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3807 value = 3.5545 dict = 1.00 all = 3.6483 } Zip/UnZip{ index = 0.0963 value = 0.2813 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 8.963218 GB avg = 0.168 KB (by entry num) total key len = 1.466980 GB avg = 0.028 KB total ukey len = 1.016563 GB avg = 0.015 KB total ukey num = 0.066484696 Billion total entry num = 0.053292425 Billion write speed all = 0.834912669 MB/sec (with seq num) write speed all = 0.800785210 MB/sec (without seq num) 2024/11/23-12:23:24.824525 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259369: 934261 keys, 36788710 bytes, temperature: kWarm 2024/11/23-12:23:24.824623 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335804824563, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259369, "file_size": 36788710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3972198424, "table_properties": {"data_size": 30400688, "index_size": 1800000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26159308, "raw_average_key_size": 28, "raw_value_size": 108058510, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335801, "oldest_key_time": 0, "file_creation_time": 1732335801, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:24.861898 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst) /dev/shm/ToplingTemp/Topling-47532-KNthHl 2024/11/23-12:23:25.855005 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst): first pass time = 0.99's, 135.155'MB/sec 2024/11/23-12:23:27.152746 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst): old prealloc_size = 147639500, real_size = 30646592 2024/11/23-12:23:27.153356 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928044 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.62 avg-zval = 30.85 2024/11/23-12:23:27.154150 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:27.972960 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259370.sst): second pass time = 1.30's, 83.452'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3058.076'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 268.123'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36165.176'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.452'MB/sec zip pipeline throughput = 83.452'MB/sec entries = 928044 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.62 avg-zval = 30.85 usrkeys = 928044 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.62 avg-zval = 30.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928044 bytes = 3248176 } seq expand size = 6496308 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.3765 value = 3.7810 dict = 1.00 all = 3.8628 } Zip/UnZip{ index = 0.0964 value = 0.2645 dict = 1.00 all = 0.2589 } ---------------------------- total value len = 9.077947 GB avg = 0.167 KB (by entry num) total key len = 1.492966 GB avg = 0.028 KB total ukey len = 1.035124 GB avg = 0.015 KB total ukey num = 0.067412740 Billion total entry num = 0.054220469 Billion write speed all = 0.845960072 MB/sec (with seq num) write speed all = 0.811247192 MB/sec (without seq num) 2024/11/23-12:23:28.016230 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259370: 928044 keys, 34746496 bytes, temperature: kWarm 2024/11/23-12:23:28.016308 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335808016264, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259370, "file_size": 34746496, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3724695938, "largest_seqno": 3972198209, "table_properties": {"data_size": 28625648, "index_size": 1788736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985232, "raw_average_key_size": 28, "raw_value_size": 108232532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928044, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335804, "oldest_key_time": 0, "file_creation_time": 1732335804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:28.053391 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst) /dev/shm/ToplingTemp/Topling-47532-Kpb2sn 2024/11/23-12:23:29.095413 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst): first pass time = 1.04's, 128.810'MB/sec 2024/11/23-12:23:30.393759 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst): old prealloc_size = 147639500, real_size = 34133552 2024/11/23-12:23:30.394355 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928023 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.63 avg-zval = 34.63 2024/11/23-12:23:30.395177 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:31.246191 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259371.sst): second pass time = 1.30's, 83.421'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.992'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3078.591'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 295.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34392.355'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.421'MB/sec zip pipeline throughput = 83.421'MB/sec entries = 928023 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 34.63 usrkeys = 928023 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 34.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928023 bytes = 3364096 } seq expand size = 6496161 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5128 value = 3.3680 dict = 1.00 all = 3.4997 } Zip/UnZip{ index = 0.0951 value = 0.2969 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 9.192676 GB avg = 0.167 KB (by entry num) total key len = 1.518950 GB avg = 0.028 KB total ukey len = 1.053685 GB avg = 0.015 KB total ukey num = 0.068340763 Billion total entry num = 0.055148492 Billion write speed all = 0.856996523 MB/sec (with seq num) write speed all = 0.821698754 MB/sec (without seq num) 2024/11/23-12:23:31.293880 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259371: 928023 keys, 38351252 bytes, temperature: kWarm 2024/11/23-12:23:31.293954 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335811293918, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259371, "file_size": 38351252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3972198787, "table_properties": {"data_size": 32135840, "index_size": 1765504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984644, "raw_average_key_size": 28, "raw_value_size": 108233182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928023, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335808, "oldest_key_time": 0, "file_creation_time": 1732335808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:31.331464 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst) /dev/shm/ToplingTemp/Topling-47532-plL33k 2024/11/23-12:23:32.357844 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst): first pass time = 1.03's, 130.773'MB/sec 2024/11/23-12:23:33.660142 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst): old prealloc_size = 147639500, real_size = 30874240 2024/11/23-12:23:33.660697 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930627 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0289 GB avg-val = 116.22 avg-zval = 31.02 2024/11/23-12:23:33.661495 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:34.488791 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259372.sst): second pass time = 1.30's, 83.110'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.218'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.848'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 273.214'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35394.307'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.110'MB/sec zip pipeline throughput = 83.110'MB/sec entries = 930627 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 31.02 usrkeys = 930627 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 31.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930627 bytes = 3257216 } seq expand size = 6514389 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5157 value = 3.7463 dict = 1.00 all = 3.8367 } Zip/UnZip{ index = 0.0951 value = 0.2669 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 9.307351 GB avg = 0.166 KB (by entry num) total key len = 1.545008 GB avg = 0.028 KB total ukey len = 1.072297 GB avg = 0.015 KB total ukey num = 0.069271390 Billion total entry num = 0.056079119 Billion write speed all = 0.868030778 MB/sec (with seq num) write speed all = 0.832146670 MB/sec (without seq num) 2024/11/23-12:23:34.532090 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259372: 930627 keys, 34982802 bytes, temperature: kWarm 2024/11/23-12:23:34.532162 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335814532125, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259372, "file_size": 34982802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 3972198236, "table_properties": {"data_size": 28871408, "index_size": 1769984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057556, "raw_average_key_size": 28, "raw_value_size": 108160226, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335811, "oldest_key_time": 0, "file_creation_time": 1732335811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:34.575882 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst) /dev/shm/ToplingTemp/Topling-47532-8P7N0m 2024/11/23-12:23:35.489363 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst): first pass time = 0.91's, 146.936'MB/sec 2024/11/23-12:23:36.895105 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst): old prealloc_size = 147639500, real_size = 35017920 2024/11/23-12:23:36.895655 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929414 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.41 avg-zval = 35.51 2024/11/23-12:23:36.896132 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:37.671018 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259373.sst): second pass time = 1.40's, 77.031'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 48.432'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.612'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.359'MB/sec write SST data time = 0.13's, 292.951'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32385.324'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.031'MB/sec zip pipeline throughput = 77.031'MB/sec entries = 929414 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.41 avg-zval = 35.51 usrkeys = 929414 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.76 avg-zval = 35.51 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 444842 bytes = 1556960 } seq expand size = 3113894 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4453 value = 3.2780 dict = 1.00 all = 3.5718 } Zip/UnZip{ index = 0.0957 value = 0.3051 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 9.418659 GB avg = 0.165 KB (by entry num) total key len = 1.571031 GB avg = 0.028 KB total ukey len = 1.090885 GB avg = 0.016 KB total ukey num = 0.070200804 Billion total entry num = 0.057008533 Billion write speed all = 0.878791631 MB/sec (with seq num) write speed all = 0.842322089 MB/sec (without seq num) 2024/11/23-12:23:37.716501 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259373: 929414 keys, 37576859 bytes, temperature: kWarm 2024/11/23-12:23:37.716571 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335817716536, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259373, "file_size": 37576859, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3972198256, "table_properties": {"data_size": 33005808, "index_size": 1779584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023592, "raw_average_key_size": 28, "raw_value_size": 108194144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335814, "oldest_key_time": 0, "file_creation_time": 1732335814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259373, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:37.762469 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst) /dev/shm/ToplingTemp/Topling-47532-uOnUFk 2024/11/23-12:23:38.685784 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst): first pass time = 0.92's, 145.371'MB/sec 2024/11/23-12:23:40.047416 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst): old prealloc_size = 147639500, real_size = 35436144 2024/11/23-12:23:40.048066 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080053 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.27 avg-zval = 30.65 2024/11/23-12:23:40.048373 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:40.748928 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259374.sst): second pass time = 1.36's, 76.411'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 52.728'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.210'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.482'MB/sec write SST data time = 0.14's, 263.547'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.025'MB, throughput = 34210.918'MB/sec zip my value time = 1.36's, unzip length = 0.104'GB zip my value throughput = 76.411'MB/sec zip pipeline throughput = 76.411'MB/sec entries = 1080053 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.27 avg-zval = 30.65 usrkeys = 1080053 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.41 avg-zval = 30.65 TagRS{ kinds = RS_Key0_TagN bytes = 168840 } TagArray{ size = 176222 bytes = 616784 } seq expand size = 1233554 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4475 value = 3.1414 dict = 1.00 all = 3.6287 } Zip/UnZip{ index = 0.0957 value = 0.3183 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 9.523869 GB avg = 0.164 KB (by entry num) total key len = 1.601273 GB avg = 0.028 KB total ukey len = 1.112486 GB avg = 0.016 KB total ukey num = 0.071280857 Billion total entry num = 0.058088586 Billion write speed all = 0.889404106 MB/sec (with seq num) write speed all = 0.852252776 MB/sec (without seq num) 2024/11/23-12:23:40.795993 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259374: 1080053 keys, 36988034 bytes, temperature: kWarm 2024/11/23-12:23:40.796073 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335820796036, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259374, "file_size": 36988034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3415308386, "table_properties": {"data_size": 33098336, "index_size": 2067584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30241484, "raw_average_key_size": 28, "raw_value_size": 103976327, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080053, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335817, "oldest_key_time": 0, "file_creation_time": 1732335817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259374, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:40.832750 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst) /dev/shm/ToplingTemp/Topling-47532-XawSBl 2024/11/23-12:23:41.857258 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst): first pass time = 1.02's, 131.012'MB/sec 2024/11/23-12:23:43.158755 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst): old prealloc_size = 147639500, real_size = 32860032 2024/11/23-12:23:43.159340 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929046 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.47 avg-zval = 33.22 2024/11/23-12:23:43.160176 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:44.014398 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259375.sst): second pass time = 1.30's, 83.187'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.474'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.877'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 290.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35503.421'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.187'MB/sec zip pipeline throughput = 83.187'MB/sec entries = 929046 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 33.22 usrkeys = 929046 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 33.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929046 bytes = 3483936 } seq expand size = 6503322 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5141 value = 3.5063 dict = 1.00 all = 3.6083 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.638577 GB avg = 0.163 KB (by entry num) total key len = 1.627286 GB avg = 0.028 KB total ukey len = 1.131067 GB avg = 0.016 KB total ukey num = 0.072209903 Billion total entry num = 0.059017632 Billion write speed all = 0.900419047 MB/sec (with seq num) write speed all = 0.862683384 MB/sec (without seq num) 2024/11/23-12:23:44.059461 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259375: 929046 keys, 37196966 bytes, temperature: kWarm 2024/11/23-12:23:44.059531 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335824059495, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259375, "file_size": 37196966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3415308387, "largest_seqno": 3972198042, "table_properties": {"data_size": 30860336, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013288, "raw_average_key_size": 28, "raw_value_size": 108204452, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929046, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335820, "oldest_key_time": 0, "file_creation_time": 1732335820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259375, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:44.098764 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst) /dev/shm/ToplingTemp/Topling-47532-kelCqj 2024/11/23-12:23:45.124155 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst): first pass time = 1.03's, 130.899'MB/sec 2024/11/23-12:23:46.473878 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst): old prealloc_size = 147639500, real_size = 33853728 2024/11/23-12:23:46.474435 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932826 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 115.88 avg-zval = 34.14 2024/11/23-12:23:46.475275 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:47.320057 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259376.sst): second pass time = 1.35's, 80.141'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3325.298'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 300.886'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36477.354'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.141'MB/sec zip pipeline throughput = 80.141'MB/sec entries = 932826 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.88 avg-zval = 34.14 usrkeys = 932826 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.88 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932826 bytes = 3498112 } seq expand size = 6529782 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5131 value = 3.3944 dict = 1.00 all = 3.5134 } Zip/UnZip{ index = 0.0951 value = 0.2946 dict = 1.00 all = 0.2846 } ---------------------------- total value len = 9.753205 GB avg = 0.163 KB (by entry num) total key len = 1.653405 GB avg = 0.028 KB total ukey len = 1.149724 GB avg = 0.016 KB total ukey num = 0.073142729 Billion total entry num = 0.059950458 Billion write speed all = 0.911427431 MB/sec (with seq num) write speed all = 0.873105448 MB/sec (without seq num) 2024/11/23-12:23:47.366165 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259376: 932826 keys, 38202110 bytes, temperature: kWarm 2024/11/23-12:23:47.366235 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335827366199, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259376, "file_size": 38202110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3972197139, "table_properties": {"data_size": 31845776, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26119128, "raw_average_key_size": 28, "raw_value_size": 108098652, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932826, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335824, "oldest_key_time": 0, "file_creation_time": 1732335824, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259376, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:47.402656 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst) /dev/shm/ToplingTemp/Topling-47532-QLFYmm 2024/11/23-12:23:48.374566 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst): first pass time = 0.97's, 138.102'MB/sec 2024/11/23-12:23:49.628832 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst): old prealloc_size = 147639500, real_size = 31288240 2024/11/23-12:23:49.629419 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922040 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.57 avg-zval = 31.76 2024/11/23-12:23:49.630224 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:50.441112 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259377.sst): second pass time = 1.25's, 86.480'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.262'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 280.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30747.176'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.480'MB/sec zip pipeline throughput = 86.480'MB/sec entries = 922040 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.57 avg-zval = 31.76 usrkeys = 922040 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.57 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922040 bytes = 3342416 } seq expand size = 6454280 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3762 value = 3.7022 dict = 1.00 all = 3.7821 } Zip/UnZip{ index = 0.0964 value = 0.2701 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 9.868060 GB avg = 0.162 KB (by entry num) total key len = 1.679222 GB avg = 0.028 KB total ukey len = 1.168165 GB avg = 0.016 KB total ukey num = 0.074064769 Billion total entry num = 0.060872498 Billion write speed all = 0.922437573 MB/sec (with seq num) write speed all = 0.883535898 MB/sec (without seq num) 2024/11/23-12:23:50.484551 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259377: 922040 keys, 35487522 bytes, temperature: kWarm 2024/11/23-12:23:50.484620 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335830484585, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259377, "file_size": 35487522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3630197102, "largest_seqno": 3972197150, "table_properties": {"data_size": 29280352, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25817120, "raw_average_key_size": 28, "raw_value_size": 108400672, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922040, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335827, "oldest_key_time": 0, "file_creation_time": 1732335827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259377, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:50.519204 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst) /dev/shm/ToplingTemp/Topling-47532-3UuvHk 2024/11/23-12:23:51.445101 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst): first pass time = 0.93's, 144.965'MB/sec 2024/11/23-12:23:52.744539 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst): old prealloc_size = 147639500, real_size = 31907872 2024/11/23-12:23:52.745080 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936213 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0299 GB avg-val = 115.36 avg-zval = 31.92 2024/11/23-12:23:52.745876 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:53.607124 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259378.sst): second pass time = 1.30's, 83.186'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3451.787'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 285.803'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36506.730'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.186'MB/sec zip pipeline throughput = 83.186'MB/sec entries = 936213 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.36 avg-zval = 31.92 usrkeys = 936213 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.36 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936213 bytes = 3393792 } seq expand size = 6553491 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4428 value = 3.6145 dict = 1.00 all = 3.7128 } Zip/UnZip{ index = 0.0958 value = 0.2767 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 9.982617 GB avg = 0.162 KB (by entry num) total key len = 1.705436 GB avg = 0.028 KB total ukey len = 1.186889 GB avg = 0.016 KB total ukey num = 0.075000982 Billion total entry num = 0.061808711 Billion write speed all = 0.933446800 MB/sec (with seq num) write speed all = 0.893956808 MB/sec (without seq num) 2024/11/23-12:23:53.652440 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259378: 936213 keys, 36150318 bytes, temperature: kWarm 2024/11/23-12:23:53.652511 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335833652475, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259378, "file_size": 36150318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3972197767, "table_properties": {"data_size": 29880592, "index_size": 1793024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26213964, "raw_average_key_size": 28, "raw_value_size": 108003842, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936213, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335830, "oldest_key_time": 0, "file_creation_time": 1732335830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259378, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:53.690862 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst) /dev/shm/ToplingTemp/Topling-47532-yNHLhl 2024/11/23-12:23:54.712460 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst): first pass time = 1.02's, 131.385'MB/sec 2024/11/23-12:23:56.003803 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst): old prealloc_size = 147639500, real_size = 31058976 2024/11/23-12:23:56.004362 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926657 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 116.84 avg-zval = 31.36 2024/11/23-12:23:56.005183 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:23:56.830418 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259379.sst): second pass time = 1.29's, 83.934'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.813'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.493'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 276.393'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35181.833'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.934'MB/sec zip pipeline throughput = 83.934'MB/sec entries = 926657 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 31.36 usrkeys = 926657 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.84 avg-zval = 31.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926657 bytes = 3474976 } seq expand size = 6486599 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5118 value = 3.7253 dict = 1.00 all = 3.7927 } Zip/UnZip{ index = 0.0951 value = 0.2684 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 10.097375 GB avg = 0.161 KB (by entry num) total key len = 1.731383 GB avg = 0.028 KB total ukey len = 1.205422 GB avg = 0.016 KB total ukey num = 0.075927639 Billion total entry num = 0.062735368 Billion write speed all = 0.944440802 MB/sec (with seq num) write speed all = 0.904369078 MB/sec (without seq num) 2024/11/23-12:23:56.875288 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259379: 926657 keys, 35388722 bytes, temperature: kWarm 2024/11/23-12:23:56.875368 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335836875331, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259379, "file_size": 35388722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3972198528, "table_properties": {"data_size": 29064080, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946396, "raw_average_key_size": 28, "raw_value_size": 108271422, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926657, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335833, "oldest_key_time": 0, "file_creation_time": 1732335833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259379, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:23:56.910797 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst) /dev/shm/ToplingTemp/Topling-47532-e0ugrn 2024/11/23-12:23:57.950381 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst): first pass time = 1.04's, 129.112'MB/sec 2024/11/23-12:23:59.232449 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst): old prealloc_size = 147639500, real_size = 30815040 2024/11/23-12:23:59.233067 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932257 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 115.97 avg-zval = 30.90 2024/11/23-12:23:59.233860 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:24:00.062503 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259380.sst): second pass time = 1.28's, 84.379'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.222'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2977.966'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 276.242'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33787.243'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.379'MB/sec zip pipeline throughput = 84.379'MB/sec entries = 932257 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 30.90 usrkeys = 932257 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932257 bytes = 3379440 } seq expand size = 6525799 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5132 value = 3.7529 dict = 1.00 all = 3.8298 } Zip/UnZip{ index = 0.0951 value = 0.2665 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 10.212016 GB avg = 0.160 KB (by entry num) total key len = 1.757486 GB avg = 0.028 KB total ukey len = 1.224067 GB avg = 0.016 KB total ukey num = 0.076859896 Billion total entry num = 0.063667625 Billion write speed all = 0.955431597 MB/sec (with seq num) write speed all = 0.914774893 MB/sec (without seq num) 2024/11/23-12:24:00.106526 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259380: 932257 keys, 35045230 bytes, temperature: kWarm 2024/11/23-12:24:00.106600 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335840106562, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259380, "file_size": 35045230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3972197589, "table_properties": {"data_size": 28808304, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26103196, "raw_average_key_size": 28, "raw_value_size": 108114534, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932257, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335836, "oldest_key_time": 0, "file_creation_time": 1732335836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259380, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:24:00.141517 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst) /dev/shm/ToplingTemp/Topling-47532-HFKhbk 2024/11/23-12:24:01.198550 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst): first pass time = 1.06's, 126.981'MB/sec 2024/11/23-12:24:02.473371 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst): old prealloc_size = 147639500, real_size = 31365744 2024/11/23-12:24:02.473956 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930961 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.17 avg-zval = 31.53 2024/11/23-12:24:02.474815 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:24:03.316947 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259381.sst): second pass time = 1.27's, 84.892'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.163'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.14's, 261.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34684.696'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.892'MB/sec zip pipeline throughput = 84.892'MB/sec entries = 930961 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.17 avg-zval = 31.53 usrkeys = 930961 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.17 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930961 bytes = 3374752 } seq expand size = 6516727 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4443 value = 3.6849 dict = 1.00 all = 3.7710 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 10.326683 GB avg = 0.160 KB (by entry num) total key len = 1.783553 GB avg = 0.028 KB total ukey len = 1.242686 GB avg = 0.016 KB total ukey num = 0.077790857 Billion total entry num = 0.064598586 Billion write speed all = 0.966414284 MB/sec (with seq num) write speed all = 0.925173803 MB/sec (without seq num) 2024/11/23-12:24:03.361367 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259381: 930961 keys, 35592206 bytes, temperature: kWarm 2024/11/23-12:24:03.361437 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335843361400, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259381, "file_size": 35592206, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3972198814, "table_properties": {"data_size": 29350112, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066908, "raw_average_key_size": 28, "raw_value_size": 108150938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930961, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335840, "oldest_key_time": 0, "file_creation_time": 1732335840, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259381, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:24:03.424531 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst) /dev/shm/ToplingTemp/Topling-47532-JCmZJj 2024/11/23-12:24:04.273011 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst): first pass time = 0.85's, 158.193'MB/sec 2024/11/23-12:24:05.564697 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst): old prealloc_size = 147639500, real_size = 33054544 2024/11/23-12:24:05.565304 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925168 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.07 avg-zval = 33.58 2024/11/23-12:24:05.566110 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-12:24:06.423348 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259382.sst): second pass time = 1.29's, 83.906'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.534'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2967.464'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 286.149'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35509.872'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.906'MB/sec zip pipeline throughput = 83.906'MB/sec entries = 925168 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 33.58 usrkeys = 925168 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925168 bytes = 3469392 } seq expand size = 6476176 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5133 value = 3.4869 dict = 1.00 all = 3.5907 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 10.441473 GB avg = 0.159 KB (by entry num) total key len = 1.809458 GB avg = 0.028 KB total ukey len = 1.261190 GB avg = 0.016 KB total ukey num = 0.078716025 Billion total entry num = 0.065523754 Billion write speed all = 0.977399570 MB/sec (with seq num) write speed all = 0.935578818 MB/sec (without seq num) 2024/11/23-12:24:06.470380 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259382: 925168 keys, 37379404 bytes, temperature: kWarm 2024/11/23-12:24:06.470452 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335846470417, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259382, "file_size": 37379404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3972197866, "table_properties": {"data_size": 31063040, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904704, "raw_average_key_size": 28, "raw_value_size": 108313044, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925168, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335843, "oldest_key_time": 0, "file_creation_time": 1732335843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259382, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:24:06.505456 139957341054528 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst) /dev/shm/ToplingTemp/Topling-47532-Vp5Y4l 2024/11/23-12:24:06.909087 139957341054528 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst): first pass time = 0.40's, 251.361'MB/sec 2024/11/23-12:24:07.885782 139957341054528 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst): old prealloc_size = 147639500, real_size = 26961984 2024/11/23-12:24:07.886201 139957341054528 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 705577 prefix = 4 raw-key = 0.0113 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0817 GB zip-val = 0.0254 GB avg-val = 115.78 avg-zval = 36.05 2024/11/23-12:24:07.886736 139957341054528 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.101 GB 2024/11/23-12:24:08.488032 139957341054528 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00/259383.sst): second pass time = 0.98's, 83.706'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.60's, 49.291'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3384.628'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.350'MB/sec write SST data time = 0.10's, 305.262'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.820'MB, throughput = 27889.852'MB/sec zip my value time = 0.98's, unzip length = 0.082'GB zip my value throughput = 83.706'MB/sec zip pipeline throughput = 83.706'MB/sec entries = 705577 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.78 avg-zval = 36.05 usrkeys = 705577 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.96 avg-zval = 36.05 TagRS{ kinds = RS_Key0_TagN bytes = 110360 } TagArray{ size = 522580 bytes = 1959696 } seq expand size = 3658060 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0141 GB value = 0.0817 GB dict = 0.82 MB all = 0.1014 GB } __ZipSize{ index = 0.0014 GB value = 0.0254 GB dict = 0.82 MB all = 0.0297 GB } UnZip/Zip{ index = 10.4425 value = 3.2118 dict = 1.00 all = 3.4176 } Zip/UnZip{ index = 0.0958 value = 0.3113 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 10.526821 GB avg = 0.159 KB (by entry num) total key len = 1.829214 GB avg = 0.028 KB total ukey len = 1.275301 GB avg = 0.016 KB total ukey num = 0.079421602 Billion total entry num = 0.066229331 Billion write speed all = 0.985622613 MB/sec (with seq num) write speed all = 0.943358486 MB/sec (without seq num) 2024/11/23-12:24:08.527190 139957341054528 [compaction_job.cc:1948] [default] [JOB 25938] Generated table #259383: 705577 keys, 29683499 bytes, temperature: kWarm 2024/11/23-12:24:08.527259 139957341054528 EVENT_LOG_v1 {"time_micros": 1732335848527224, "cf_name": "default", "job": 25938, "event": "table_file_creation", "file_number": 259383, "file_size": 29683499, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3972198546, "table_properties": {"data_size": 25434032, "index_size": 1351360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19756156, "raw_average_key_size": 28, "raw_value_size": 81690122, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 705577, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732335846, "oldest_key_time": 0, "file_creation_time": 1732335846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 259383, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-12:24:08.556204 139957341054528 [compaction_job.cc:703] job-25938: subcompact[0], size: 0.539560 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-25938/att-00 [259369,259370,259371,259372,259373,259374,259375,259376,259377,259378,259379,259380,259381,259382,259383] 2024/11/23-12:24:08.647886 139957341054528 (Original Log Time 2024/11/23-12:23:21.239740) EVENT_LOG_v1 {"time_micros": 1732335801239699, "job": 25938, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [259294], "files_L3": [259366, 259254, 259255, 259256, 259257, 259258, 259259, 259260, 259261, 259262, 259263, 259264, 259265, 259234, 259235, 259346], "score": 0.998773, "input_data_size": 569287922, "oldest_snapshot_seqno": 3972505410} 2024/11/23-12:24:08.647889 139957341054528 (Original Log Time 2024/11/23-12:24:08.645612) [compaction_job.cc:2022] [default] [JOB 25938] Compacted 1@2 + 16@3 files to L3 => 539560130 bytes 2024/11/23-12:24:08.647891 139957341054528 (Original Log Time 2024/11/23-12:24:08.647822) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.4 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 515.8 +0.0 blob) out(514.6 +0.0 blob), read-write-amplify(39.0) write-amplify(19.0) OK, records in: 13915972, records dropped: 44805 output_compression: Snappy 2024/11/23-12:24:08.647897 139957341054528 (Original Log Time 2024/11/23-12:24:08.647865) EVENT_LOG_v1 {"time_micros": 1732335848647844, "cf": "default", "job": 25938, "event": "compaction_finished", "compaction_time_micros": 47316471, "compaction_time_cpu_micros": 20345164, "output_level": 3, "num_output_files": 15, "total_output_size": 539560130, "num_input_records": 13915972, "num_output_records": 13871167, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 365382487, "file_range_sync_nanos": 0, "file_fsync_nanos": 10464, "file_prepare_write_nanos": 45075, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-12:24:08.793390 139957341054528 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732335801,"executesMs":47553,"compactionJobId":25938,"attempt":0,"compactionInputRawBytes":1986857585,"compactionInputZipBytes":569287922,"compactionOutputRawBytes":0,"compactionOutputZipBytes":539560130,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-2aaa0d2f7f112232af62ef4e4cfc6c6d-ee97e97fddcb115f-00"}}, response = 2024/11/23-12:24:08.793479 139957341054528 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-25938/att-00: olev 3, work 47.554 s, result 7.680 ms, install 2.304 ms, input{raw 1.850 GiB zip 542.915 MiB}