2024/11/23-00:31:31.354648 140024869324352 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00: fork to child time = 0.009265 sec 2024/11/23-00:31:31.355474 140024869324352 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00 2024/11/23-00:31:31.362932 140024869324352 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00 2024/11/23-00:31:31.364067 140024869324352 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-00:31:31.364339 140024869324352 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-00:31:31.409114 140024869324352 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15289/att-00/MANIFEST-206347 2024/11/23-00:31:31.438857 140024869324352 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15289/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 236461, last_sequence is 3676991076, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 236254 2024/11/23-00:31:31.438864 140024869324352 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-00:31:31.438956 140024869324352 [version_set.cc:5721] Creating manifest 236461 2024/11/23-00:31:31.468732 140024869324352 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-00:31:31.496728 140024869324352 [compaction_job.cc:2437] [default] [JOB 15289] Compacting 1@2 + 13@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-00:31:31.496745 140024869324352 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11455 Base level 2, inputs: [236399(27MB)], [236458(35MB) 236459(5993KB) 236369(35MB) 236370(35MB) 236371(33MB) 236372(35MB) 236373(36MB) 236374(19MB) 236317(34MB) 236318(35MB) 236319(35MB) 236320(35MB) 236430(34MB)] 2024/11/23-00:31:31.503003 140024869324352 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:31:31.503035 140024869324352 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:31:31.503062 140024869324352 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:31:31.503067 140024869324352 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:31:31.503076 140024869324352 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:31:31.960716 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst) /dev/shm/ToplingTemp/Topling-351427-9sXIlt 2024/11/23-00:31:32.544502 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst): first pass time = 0.58's, 229.928'MB/sec 2024/11/23-00:31:34.291691 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst): old prealloc_size = 147639500, real_size = 33007056 2024/11/23-00:31:34.292335 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925976 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.95 avg-zval = 33.49 2024/11/23-00:31:34.293204 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:35.466427 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236462.sst): second pass time = 1.75's, 62.015'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 31.682'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2831.519'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.230'MB/sec write SST data time = 0.17's, 219.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30695.610'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 62.015'MB/sec zip pipeline throughput = 79.264'MB/sec entries = 925976 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.49 usrkeys = 925976 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925976 bytes = 3356672 } seq expand size = 6481832 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5133 value = 3.4917 dict = 1.00 all = 3.6062 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 0.114772 GB avg = 0.124 KB (by entry num) total key len = 0.025927 GB avg = 0.028 KB total ukey len = 0.018520 GB avg = 0.020 KB total ukey num = 0.000925976 Billion total entry num = 0.000925976 Billion write speed all = 0.002496783 MB/sec (with seq num) write speed all = 0.002365328 MB/sec (without seq num) 2024/11/23-00:31:35.502461 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236462: 925976 keys, 37218396 bytes, temperature: kWarm 2024/11/23-00:31:35.502547 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293095502495, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236462, "file_size": 37218396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3676420923, "table_properties": {"data_size": 31013824, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927328, "raw_average_key_size": 28, "raw_value_size": 108290460, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925976, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293091, "oldest_key_time": 0, "file_creation_time": 1732293091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236462, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:35.538393 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst) /dev/shm/ToplingTemp/Topling-351427-WCDlfu 2024/11/23-00:31:36.564595 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst): first pass time = 1.03's, 130.795'MB/sec 2024/11/23-00:31:38.415648 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst): old prealloc_size = 147639500, real_size = 32792960 2024/11/23-00:31:38.416231 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931490 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.09 avg-zval = 33.05 2024/11/23-00:31:38.417007 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:39.376069 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236463.sst): second pass time = 1.85's, 58.437'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.547'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3136.839'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 288.956'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33121.927'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.437'MB/sec zip pipeline throughput = 105.898'MB/sec entries = 931490 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.05 usrkeys = 931490 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931490 bytes = 3376672 } seq expand size = 6520430 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5132 value = 3.5123 dict = 1.00 all = 3.6255 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 0.229429 GB avg = 0.124 KB (by entry num) total key len = 0.052009 GB avg = 0.028 KB total ukey len = 0.037149 GB avg = 0.020 KB total ukey num = 0.001857466 Billion total entry num = 0.001857466 Billion write speed all = 0.004993905 MB/sec (with seq num) write speed all = 0.004730230 MB/sec (without seq num) 2024/11/23-00:31:39.408169 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236463: 931490 keys, 37020528 bytes, temperature: kWarm 2024/11/23-00:31:39.408230 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293099408198, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236463, "file_size": 37020528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3676422226, "table_properties": {"data_size": 30787888, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081720, "raw_average_key_size": 28, "raw_value_size": 108136136, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931490, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293095, "oldest_key_time": 0, "file_creation_time": 1732293095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236463, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:39.445967 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst) /dev/shm/ToplingTemp/Topling-351427-Yak9ns 2024/11/23-00:31:40.305190 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst): first pass time = 0.86's, 156.217'MB/sec 2024/11/23-00:31:41.729902 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst): old prealloc_size = 147639500, real_size = 33315280 2024/11/23-00:31:41.730456 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929784 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.35 avg-zval = 33.68 2024/11/23-00:31:41.731254 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:42.673939 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236464.sst): second pass time = 1.42's, 75.979'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.765'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.585'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.13's, 293.977'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33330.451'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.979'MB/sec zip pipeline throughput = 75.979'MB/sec entries = 929784 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 33.68 usrkeys = 929784 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929784 bytes = 3370480 } seq expand size = 6508488 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5130 value = 3.4548 dict = 1.00 all = 3.5755 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.344121 GB avg = 0.123 KB (by entry num) total key len = 0.078043 GB avg = 0.028 KB total ukey len = 0.055745 GB avg = 0.020 KB total ukey num = 0.002787250 Billion total entry num = 0.002787250 Billion write speed all = 0.007490549 MB/sec (with seq num) write speed all = 0.007094911 MB/sec (without seq num) 2024/11/23-00:31:42.708020 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236464: 929784 keys, 37538528 bytes, temperature: kWarm 2024/11/23-00:31:42.708082 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293102708051, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236464, "file_size": 37538528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3676422555, "table_properties": {"data_size": 31313792, "index_size": 1768832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033952, "raw_average_key_size": 28, "raw_value_size": 108183828, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929784, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293099, "oldest_key_time": 0, "file_creation_time": 1732293099, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236464, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:42.744950 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst) /dev/shm/ToplingTemp/Topling-351427-PlfDgr 2024/11/23-00:31:43.582409 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst): first pass time = 0.84's, 160.275'MB/sec 2024/11/23-00:31:45.022660 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst): old prealloc_size = 147639500, real_size = 32625248 2024/11/23-00:31:45.023204 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923526 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.33 avg-zval = 33.17 2024/11/23-00:31:45.025009 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:45.938804 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236465.sst): second pass time = 1.44's, 75.284'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.203'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.385'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 285.166'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33420.843'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.284'MB/sec zip pipeline throughput = 75.284'MB/sec entries = 923526 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 33.17 usrkeys = 923526 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923526 bytes = 3347792 } seq expand size = 6464682 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5133 value = 3.5368 dict = 1.00 all = 3.6442 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 0.458945 GB avg = 0.124 KB (by entry num) total key len = 0.103902 GB avg = 0.028 KB total ukey len = 0.074216 GB avg = 0.020 KB total ukey num = 0.003710776 Billion total entry num = 0.003710776 Billion write speed all = 0.009986128 MB/sec (with seq num) write speed all = 0.009459430 MB/sec (without seq num) 2024/11/23-00:31:45.971287 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236465: 923526 keys, 36830318 bytes, temperature: kWarm 2024/11/23-00:31:45.971348 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293105971316, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236465, "file_size": 36830318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3676422103, "table_properties": {"data_size": 30637328, "index_size": 1756864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25858728, "raw_average_key_size": 28, "raw_value_size": 108359004, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293102, "oldest_key_time": 0, "file_creation_time": 1732293102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236465, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:46.006753 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst) /dev/shm/ToplingTemp/Topling-351427-KiZ6Jq 2024/11/23-00:31:46.799970 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst): first pass time = 0.79's, 169.215'MB/sec 2024/11/23-00:31:48.131382 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst): old prealloc_size = 147639500, real_size = 33156448 2024/11/23-00:31:48.131946 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932779 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0311 GB avg-val = 115.89 avg-zval = 33.39 2024/11/23-00:31:48.132702 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:49.056107 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236466.sst): second pass time = 1.33's, 81.260'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.306'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.406'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.293'MB/sec write SST data time = 0.13's, 291.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32406.364'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.260'MB/sec zip pipeline throughput = 81.260'MB/sec entries = 932779 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.89 avg-zval = 33.39 usrkeys = 932779 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.89 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932779 bytes = 3264736 } seq expand size = 6529453 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5141 value = 3.4704 dict = 1.00 all = 3.6011 } Zip/UnZip{ index = 0.0951 value = 0.2881 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 0.573574 GB avg = 0.124 KB (by entry num) total key len = 0.130020 GB avg = 0.028 KB total ukey len = 0.092871 GB avg = 0.020 KB total ukey num = 0.004643555 Billion total entry num = 0.004643555 Billion write speed all = 0.012482603 MB/sec (with seq num) write speed all = 0.011823545 MB/sec (without seq num) 2024/11/23-00:31:49.088428 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236466: 932779 keys, 37271352 bytes, temperature: kWarm 2024/11/23-00:31:49.088492 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293109088458, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236466, "file_size": 37271352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3676422832, "table_properties": {"data_size": 31148752, "index_size": 1774336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26117812, "raw_average_key_size": 28, "raw_value_size": 108100050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932779, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293105, "oldest_key_time": 0, "file_creation_time": 1732293105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236466, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:49.125936 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst) /dev/shm/ToplingTemp/Topling-351427-jOK0Yr 2024/11/23-00:31:49.911764 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst): first pass time = 0.79's, 170.806'MB/sec 2024/11/23-00:31:51.217018 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst): old prealloc_size = 147639500, real_size = 33135536 2024/11/23-00:31:51.217549 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929344 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.42 avg-zval = 33.50 2024/11/23-00:31:51.218276 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:52.102327 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236467.sst): second pass time = 1.30's, 82.963'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.721'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3475.987'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 281.103'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33035.074'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.963'MB/sec zip pipeline throughput = 82.963'MB/sec entries = 929344 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.50 usrkeys = 929344 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.23 avg-zval = 33.50 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 771635 bytes = 2797184 } seq expand size = 5401445 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5152 value = 3.4750 dict = 1.00 all = 3.6339 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 0.687172 GB avg = 0.123 KB (by entry num) total key len = 0.156041 GB avg = 0.028 KB total ukey len = 0.111458 GB avg = 0.020 KB total ukey num = 0.005572899 Billion total entry num = 0.005572899 Billion write speed all = 0.014958808 MB/sec (with seq num) write speed all = 0.014167891 MB/sec (without seq num) 2024/11/23-00:31:52.136689 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236467: 929344 keys, 36934892 bytes, temperature: kWarm 2024/11/23-00:31:52.136752 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293112136719, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236467, "file_size": 36934892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3676423129, "table_properties": {"data_size": 31135392, "index_size": 1767616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021632, "raw_average_key_size": 28, "raw_value_size": 108196152, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929344, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293109, "oldest_key_time": 0, "file_creation_time": 1732293109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236467, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:52.172743 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst) /dev/shm/ToplingTemp/Topling-351427-2Qy0ut 2024/11/23-00:31:53.277682 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst): first pass time = 1.10's, 121.476'MB/sec 2024/11/23-00:31:54.604397 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst): old prealloc_size = 147639500, real_size = 32291664 2024/11/23-00:31:54.604960 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926356 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.89 avg-zval = 32.69 2024/11/23-00:31:54.605725 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:55.479454 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236468.sst): second pass time = 1.33's, 81.701'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.782'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.553'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 277.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31389.639'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.701'MB/sec zip pipeline throughput = 81.701'MB/sec entries = 926356 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.89 avg-zval = 32.69 usrkeys = 926356 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.70 avg-zval = 32.69 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 901068 bytes = 3266384 } seq expand size = 6307476 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4489 value = 3.5752 dict = 1.00 all = 3.6710 } Zip/UnZip{ index = 0.0957 value = 0.2797 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 0.801759 GB avg = 0.123 KB (by entry num) total key len = 0.181979 GB avg = 0.028 KB total ukey len = 0.129985 GB avg = 0.020 KB total ukey num = 0.006499255 Billion total entry num = 0.006499255 Billion write speed all = 0.017450715 MB/sec (with seq num) write speed all = 0.016528383 MB/sec (without seq num) 2024/11/23-00:31:55.518940 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236468: 926356 keys, 36561248 bytes, temperature: kWarm 2024/11/23-00:31:55.519002 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293115518971, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236468, "file_size": 36561248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3676422587, "table_properties": {"data_size": 30286784, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937968, "raw_average_key_size": 28, "raw_value_size": 108279800, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926356, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293112, "oldest_key_time": 0, "file_creation_time": 1732293112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236468, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:55.556156 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst) /dev/shm/ToplingTemp/Topling-351427-YWrhzt 2024/11/23-00:31:56.385892 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst): first pass time = 0.83's, 161.768'MB/sec 2024/11/23-00:31:57.686520 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst): old prealloc_size = 147639500, real_size = 31212400 2024/11/23-00:31:57.687083 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921873 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.59 avg-zval = 31.69 2024/11/23-00:31:57.687842 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:31:58.541053 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236469.sst): second pass time = 1.30's, 83.422'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.305'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3272.860'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 278.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32052.302'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.422'MB/sec zip pipeline throughput = 83.422'MB/sec entries = 921873 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.59 avg-zval = 31.69 usrkeys = 921873 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.59 avg-zval = 31.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921873 bytes = 3226576 } seq expand size = 6453111 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.7104 dict = 1.00 all = 3.8026 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 0.916618 GB avg = 0.124 KB (by entry num) total key len = 0.207792 GB avg = 0.028 KB total ukey len = 0.148423 GB avg = 0.020 KB total ukey num = 0.007421128 Billion total entry num = 0.007421128 Billion write speed all = 0.019945019 MB/sec (with seq num) write speed all = 0.018891918 MB/sec (without seq num) 2024/11/23-00:31:58.581021 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236469: 921873 keys, 35295992 bytes, temperature: kWarm 2024/11/23-00:31:58.581095 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293118581050, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236469, "file_size": 35295992, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3517424421, "largest_seqno": 3676422601, "table_properties": {"data_size": 29216416, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812444, "raw_average_key_size": 28, "raw_value_size": 108405366, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921873, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293115, "oldest_key_time": 0, "file_creation_time": 1732293115, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236469, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:31:58.614803 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst) /dev/shm/ToplingTemp/Topling-351427-vXLjlt 2024/11/23-00:31:59.485965 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst): first pass time = 0.87's, 154.074'MB/sec 2024/11/23-00:32:00.813681 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst): old prealloc_size = 147639500, real_size = 32360192 2024/11/23-00:32:00.814242 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926241 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.91 avg-zval = 32.78 2024/11/23-00:32:00.815009 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:32:01.718158 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236470.sst): second pass time = 1.33's, 81.594'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.307'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3272.446'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 287.117'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35382.604'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.594'MB/sec zip pipeline throughput = 81.594'MB/sec entries = 926241 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 32.78 usrkeys = 926241 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 32.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926241 bytes = 3241856 } seq expand size = 6483687 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5125 value = 3.5659 dict = 1.00 all = 3.6816 } Zip/UnZip{ index = 0.0951 value = 0.2804 dict = 1.00 all = 0.2716 } ---------------------------- total value len = 1.031385 GB avg = 0.124 KB (by entry num) total key len = 0.233726 GB avg = 0.028 KB total ukey len = 0.166947 GB avg = 0.020 KB total ukey num = 0.008347369 Billion total entry num = 0.008347369 Billion write speed all = 0.022439549 MB/sec (with seq num) write speed all = 0.021255076 MB/sec (without seq num) 2024/11/23-00:32:01.752037 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236470: 926241 keys, 36456390 bytes, temperature: kWarm 2024/11/23-00:32:01.752097 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293121752067, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236470, "file_size": 36456390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3676423580, "table_properties": {"data_size": 30366256, "index_size": 1762176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934748, "raw_average_key_size": 28, "raw_value_size": 108283114, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926241, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293118, "oldest_key_time": 0, "file_creation_time": 1732293118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236470, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:32:01.814407 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst) /dev/shm/ToplingTemp/Topling-351427-qwYe9t 2024/11/23-00:32:02.656737 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst): first pass time = 0.84's, 159.349'MB/sec 2024/11/23-00:32:04.007970 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst): old prealloc_size = 147639500, real_size = 33481056 2024/11/23-00:32:04.008524 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929819 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.35 avg-zval = 33.86 2024/11/23-00:32:04.009306 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:32:04.922442 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236471.sst): second pass time = 1.35's, 80.101'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.104'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.678'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 291.982'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31643.153'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.101'MB/sec zip pipeline throughput = 80.101'MB/sec entries = 929819 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 33.86 usrkeys = 929819 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929819 bytes = 3254384 } seq expand size = 6508733 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5137 value = 3.4366 dict = 1.00 all = 3.5708 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 1.146076 GB avg = 0.124 KB (by entry num) total key len = 0.259761 GB avg = 0.028 KB total ukey len = 0.185544 GB avg = 0.020 KB total ukey num = 0.009277188 Billion total entry num = 0.009277188 Billion write speed all = 0.024934229 MB/sec (with seq num) write speed all = 0.023617892 MB/sec (without seq num) 2024/11/23-00:32:04.955096 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236471: 929819 keys, 37587562 bytes, temperature: kWarm 2024/11/23-00:32:04.955161 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293124955129, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236471, "file_size": 37587562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3676423741, "table_properties": {"data_size": 31479632, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034932, "raw_average_key_size": 28, "raw_value_size": 108182862, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929819, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293121, "oldest_key_time": 0, "file_creation_time": 1732293121, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236471, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:32:05.001911 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst) /dev/shm/ToplingTemp/Topling-351427-9B2BUs 2024/11/23-00:32:05.829429 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst): first pass time = 0.83's, 162.201'MB/sec 2024/11/23-00:32:07.140886 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst): old prealloc_size = 147639500, real_size = 33802608 2024/11/23-00:32:07.141428 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932101 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 115.99 avg-zval = 34.11 2024/11/23-00:32:07.142206 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:32:08.060857 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236472.sst): second pass time = 1.31's, 82.486'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.547'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 295.972'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25694.450'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.486'MB/sec zip pipeline throughput = 82.486'MB/sec entries = 932101 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 34.11 usrkeys = 932101 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 34.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932101 bytes = 3378880 } seq expand size = 6524707 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5110 value = 3.4004 dict = 1.00 all = 3.5291 } Zip/UnZip{ index = 0.0951 value = 0.2941 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 1.260720 GB avg = 0.123 KB (by entry num) total key len = 0.285860 GB avg = 0.028 KB total ukey len = 0.204186 GB avg = 0.020 KB total ukey num = 0.010209289 Billion total entry num = 0.010209289 Billion write speed all = 0.027428942 MB/sec (with seq num) write speed all = 0.025980429 MB/sec (without seq num) 2024/11/23-00:32:08.095256 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236472: 932101 keys, 38032048 bytes, temperature: kWarm 2024/11/23-00:32:08.095320 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293128095288, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236472, "file_size": 38032048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3676423537, "table_properties": {"data_size": 31795808, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26098828, "raw_average_key_size": 28, "raw_value_size": 108118994, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932101, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293124, "oldest_key_time": 0, "file_creation_time": 1732293124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:32:08.130668 140024869324352 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst) /dev/shm/ToplingTemp/Topling-351427-mvZMHq 2024/11/23-00:32:08.587238 140024869324352 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst): first pass time = 0.46's, 249.076'MB/sec 2024/11/23-00:32:09.755941 140024869324352 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst): old prealloc_size = 147639500, real_size = 27955328 2024/11/23-00:32:09.756397 140024869324352 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 785555 prefix = 4 raw-key = 0.0126 GB zip-key = 0.0015 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0917 GB zip-val = 0.0263 GB avg-val = 116.75 avg-zval = 33.43 2024/11/23-00:32:09.757049 140024869324352 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0020 GB, waited 0.000 sec, Key+Value = 0.114 GB 2024/11/23-00:32:10.545546 140024869324352 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00/236473.sst): second pass time = 1.17's, 78.545'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 39.684'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3444.283'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.289'MB/sec write SST data time = 0.11's, 288.726'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.918'MB, throughput = 27003.911'MB/sec zip my value time = 1.17's, unzip length = 0.092'GB zip my value throughput = 78.545'MB/sec zip pipeline throughput = 78.545'MB/sec entries = 785555 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.43 usrkeys = 785555 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 785555 bytes = 2651264 } seq expand size = 5498885 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0157 GB value = 0.0917 GB dict = 0.92 MB all = 0.1137 GB } __ZipSize{ index = 0.0015 GB value = 0.0263 GB dict = 0.92 MB all = 0.0313 GB } UnZip/Zip{ index = 10.5097 value = 3.4921 dict = 1.00 all = 3.6290 } Zip/UnZip{ index = 0.0952 value = 0.2864 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 1.357934 GB avg = 0.124 KB (by entry num) total key len = 0.307856 GB avg = 0.028 KB total ukey len = 0.219897 GB avg = 0.020 KB total ukey num = 0.010994844 Billion total entry num = 0.010994844 Billion write speed all = 0.029541856 MB/sec (with seq num) write speed all = 0.027981957 MB/sec (without seq num) 2024/11/23-00:32:10.577154 140024869324352 [compaction_job.cc:1948] [default] [JOB 15289] Generated table #236473: 785555 keys, 31333698 bytes, temperature: kWarm 2024/11/23-00:32:10.577225 140024869324352 EVENT_LOG_v1 {"time_micros": 1732293130577186, "cf_name": "default", "job": 15289, "event": "table_file_creation", "file_number": 236473, "file_size": 31333698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3553069460, "largest_seqno": 3676423226, "table_properties": {"data_size": 26263856, "index_size": 1494912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 21995540, "raw_average_key_size": 28, "raw_value_size": 91715610, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 785555, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293128, "oldest_key_time": 0, "file_creation_time": 1732293128, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:32:10.601563 140024869324352 [compaction_job.cc:703] job-15289: subcompact[0], size: 0.438081 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15289/att-00 [236462,236463,236464,236465,236466,236467,236468,236469,236470,236471,236472,236473] 2024/11/23-00:32:10.680237 140024869324352 (Original Log Time 2024/11/23-00:31:31.496793) EVENT_LOG_v1 {"time_micros": 1732293091496756, "job": 15289, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [236399], "files_L3": [236458, 236459, 236369, 236370, 236371, 236372, 236373, 236374, 236317, 236318, 236319, 236320, 236430], "score": 0.999787, "input_data_size": 462316700, "oldest_snapshot_seqno": 3676991076} 2024/11/23-00:32:10.680240 140024869324352 (Original Log Time 2024/11/23-00:32:10.679579) [compaction_job.cc:2022] [default] [JOB 15289] Compacted 1@2 + 13@3 files to L3 => 438080952 bytes 2024/11/23-00:32:10.680242 140024869324352 (Original Log Time 2024/11/23-00:32:10.680170) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.2 wr, level 3, files in(1, 13) out(12 +0 blob) MB in(27.1, 413.8 +0.0 blob) out(417.8 +0.0 blob), read-write-amplify(31.7) write-amplify(15.4) OK, records in: 11078398, records dropped: 83554 output_compression: Snappy 2024/11/23-00:32:10.680247 140024869324352 (Original Log Time 2024/11/23-00:32:10.680217) EVENT_LOG_v1 {"time_micros": 1732293130680193, "cf": "default", "job": 15289, "event": "compaction_finished", "compaction_time_micros": 39104780, "compaction_time_cpu_micros": 17567228, "output_level": 3, "num_output_files": 12, "total_output_size": 438080952, "num_input_records": 11078398, "num_output_records": 10994844, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 280088562, "file_range_sync_nanos": 0, "file_fsync_nanos": 10815, "file_prepare_write_nanos": 25047, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-00:32:10.822941 140024869324352 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732293091,"executesMs":39318,"compactionJobId":15289,"attempt":0,"compactionInputRawBytes":1601971676,"compactionInputZipBytes":462316700,"compactionOutputRawBytes":0,"compactionOutputZipBytes":438080952,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-07bc02b2802b4d84ead615fa2ebbefdb-f0ada16173fffcec-00"}}, response = 2024/11/23-00:32:10.823018 140024869324352 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15289/att-00: olev 3, work 39.318 s, result 6.483 ms, install 0.686 ms, input{raw 1.492 GiB zip 440.900 MiB}