2024/11/23-20:02:55.123615 139957223556672 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00: fork to child time = 0.022750 sec 2024/11/23-20:02:55.124621 139957223556672 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00 2024/11/23-20:02:55.130997 139957223556672 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00 2024/11/23-20:02:55.132238 139957223556672 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-20:02:55.132596 139957223556672 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-20:02:55.191887 139957223556672 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34771/att-00/MANIFEST-206347 2024/11/23-20:02:55.207735 139957223556672 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34771/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278429, last_sequence is 4232747101, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 278281 2024/11/23-20:02:55.207746 139957223556672 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-20:02:55.207870 139957223556672 [version_set.cc:5721] Creating manifest 278429 2024/11/23-20:02:55.241906 139957223556672 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-20:02:55.261420 139957223556672 [compaction_job.cc:2437] [default] [JOB 34771] Compacting 1@2 + 7@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-20:02:55.261446 139957223556672 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26029 Base level 2, inputs: [278407(27MB)], [278378(36MB) 278379(33MB) 278380(10MB) 278336(35MB) 278337(35MB) 278338(35MB) 278419(29MB)] 2024/11/23-20:02:55.270826 139957223556672 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:02:55.270880 139957223556672 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:02:55.270909 139957223556672 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:02:55.270918 139957223556672 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:02:55.270935 139957223556672 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:02:56.050710 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst) /dev/shm/ToplingTemp/Topling-174028-ggQ2Kl 2024/11/23-20:02:56.972589 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst): first pass time = 0.92's, 145.600'MB/sec 2024/11/23-20:02:58.482915 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst): old prealloc_size = 147639500, real_size = 34333136 2024/11/23-20:02:58.483651 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929203 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.44 avg-zval = 34.80 2024/11/23-20:02:58.484514 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:02:59.635144 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278430.sst): second pass time = 1.51's, 71.690'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 33.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2491.013'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.32's, 120.336'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34967.073'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.690'MB/sec zip pipeline throughput = 71.690'MB/sec entries = 929203 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.44 avg-zval = 34.80 usrkeys = 929203 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.44 avg-zval = 34.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929203 bytes = 3136080 } seq expand size = 6504421 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5121 value = 3.3464 dict = 1.00 all = 3.5021 } Zip/UnZip{ index = 0.0951 value = 0.2988 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 8.963324 GB avg = 0.168 KB (by entry num) total key len = 1.466839 GB avg = 0.028 KB total ukey len = 1.016462 GB avg = 0.015 KB total ukey num = 0.066479638 Billion total entry num = 0.053287367 Billion write speed all = 0.260315311 MB/sec (with seq num) write speed all = 0.249675771 MB/sec (without seq num) 2024/11/23-20:02:59.688156 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278430: 929203 keys, 38325424 bytes, temperature: kWarm 2024/11/23-20:02:59.688285 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363379688204, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278430, "file_size": 38325424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4159804009, "largest_seqno": 4232676573, "table_properties": {"data_size": 32332800, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26017684, "raw_average_key_size": 28, "raw_value_size": 108200050, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929203, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363376, "oldest_key_time": 0, "file_creation_time": 1732363376, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:02:59.771486 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst) /dev/shm/ToplingTemp/Topling-174028-BqrQvm 2024/11/23-20:03:01.213117 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst): first pass time = 1.44's, 93.112'MB/sec 2024/11/23-20:03:02.948310 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst): old prealloc_size = 147639500, real_size = 31216192 2024/11/23-20:03:02.949013 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931901 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 116.03 avg-zval = 31.34 2024/11/23-20:03:02.949851 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:04.283766 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278431.sst): second pass time = 1.73's, 62.340'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.34's, 26.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2619.364'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.228'MB/sec write SST data time = 0.31's, 113.210'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34755.948'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.340'MB/sec zip pipeline throughput = 62.340'MB/sec entries = 931901 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 31.34 usrkeys = 931901 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 31.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931901 bytes = 3145184 } seq expand size = 6523307 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5110 value = 3.7016 dict = 1.00 all = 3.8117 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2624 } ---------------------------- total value len = 9.077972 GB avg = 0.167 KB (by entry num) total key len = 1.492932 GB avg = 0.028 KB total ukey len = 1.035100 GB avg = 0.015 KB total ukey num = 0.067411539 Billion total entry num = 0.054219268 Billion write speed all = 0.263797313 MB/sec (with seq num) write speed all = 0.252972962 MB/sec (without seq num) 2024/11/23-20:03:04.331744 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278431: 931901 keys, 35212284 bytes, temperature: kWarm 2024/11/23-20:03:04.331837 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363384331790, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278431, "file_size": 35212284, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223986, "largest_seqno": 4232676102, "table_properties": {"data_size": 29209840, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26093228, "raw_average_key_size": 28, "raw_value_size": 108124590, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931901, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363379, "oldest_key_time": 0, "file_creation_time": 1732363379, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:04.392735 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst) /dev/shm/ToplingTemp/Topling-174028-dKGKTn 2024/11/23-20:03:06.064782 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst): first pass time = 1.67's, 80.274'MB/sec 2024/11/23-20:03:07.884066 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst): old prealloc_size = 147639500, real_size = 32356608 2024/11/23-20:03:07.884615 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927201 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.76 avg-zval = 32.74 2024/11/23-20:03:07.885362 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:08.735964 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278432.sst): second pass time = 1.82's, 59.545'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.660'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.505'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 286.475'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35932.562'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.545'MB/sec zip pipeline throughput = 59.545'MB/sec entries = 927201 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 32.74 usrkeys = 927201 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 32.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927201 bytes = 3129312 } seq expand size = 6490407 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5142 value = 3.5656 dict = 1.00 all = 3.6933 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2708 } ---------------------------- total value len = 9.192719 GB avg = 0.167 KB (by entry num) total key len = 1.518894 GB avg = 0.028 KB total ukey len = 1.053644 GB avg = 0.015 KB total ukey num = 0.068338740 Billion total entry num = 0.055146469 Billion write speed all = 0.267278997 MB/sec (with seq num) write speed all = 0.256270762 MB/sec (without seq num) 2024/11/23-20:03:08.781440 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278432: 927201 keys, 36340484 bytes, temperature: kWarm 2024/11/23-20:03:08.781511 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363388781477, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278432, "file_size": 36340484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4142233521, "largest_seqno": 4232676821, "table_properties": {"data_size": 30360880, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961628, "raw_average_key_size": 28, "raw_value_size": 108256222, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927201, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363384, "oldest_key_time": 0, "file_creation_time": 1732363384, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:08.818274 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst) /dev/shm/ToplingTemp/Topling-174028-MieKHn 2024/11/23-20:03:09.930538 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst): first pass time = 1.11's, 120.676'MB/sec 2024/11/23-20:03:11.422844 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst): old prealloc_size = 147639500, real_size = 33602576 2024/11/23-20:03:11.423443 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932238 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0316 GB avg-val = 115.97 avg-zval = 33.89 2024/11/23-20:03:11.424212 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:12.317832 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278433.sst): second pass time = 1.49's, 72.487'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.012'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.074'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.304'MB/sec write SST data time = 0.13's, 291.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34400.217'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.487'MB/sec zip pipeline throughput = 72.487'MB/sec entries = 932238 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 33.89 usrkeys = 932238 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 33.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932238 bytes = 3146320 } seq expand size = 6525666 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5133 value = 3.4218 dict = 1.00 all = 3.5697 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 9.307360 GB avg = 0.166 KB (by entry num) total key len = 1.544996 GB avg = 0.028 KB total ukey len = 1.072289 GB avg = 0.015 KB total ukey num = 0.069270978 Billion total entry num = 0.056078707 Billion write speed all = 0.270766663 MB/sec (with seq num) write speed all = 0.259573338 MB/sec (without seq num) 2024/11/23-20:03:12.364566 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278433: 932238 keys, 37599542 bytes, temperature: kWarm 2024/11/23-20:03:12.364653 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363392364601, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278433, "file_size": 37599542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4152787506, "largest_seqno": 4232676505, "table_properties": {"data_size": 31595904, "index_size": 1773440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102664, "raw_average_key_size": 28, "raw_value_size": 108115068, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363388, "oldest_key_time": 0, "file_creation_time": 1732363388, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278433, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:12.404070 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst) /dev/shm/ToplingTemp/Topling-174028-vcTbRl 2024/11/23-20:03:13.417695 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst): first pass time = 1.01's, 132.419'MB/sec 2024/11/23-20:03:14.922330 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst): old prealloc_size = 147639500, real_size = 31872576 2024/11/23-20:03:14.922899 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925868 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.96 avg-zval = 32.27 2024/11/23-20:03:14.923680 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:15.782032 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278434.sst): second pass time = 1.50's, 72.035'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.480'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 275.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35036.050'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.035'MB/sec zip pipeline throughput = 72.035'MB/sec entries = 925868 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 32.27 usrkeys = 925868 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925868 bytes = 3124816 } seq expand size = 6481076 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5113 value = 3.6244 dict = 1.00 all = 3.7436 } Zip/UnZip{ index = 0.0951 value = 0.2759 dict = 1.00 all = 0.2671 } ---------------------------- total value len = 9.422134 GB avg = 0.165 KB (by entry num) total key len = 1.570921 GB avg = 0.028 KB total ukey len = 1.090806 GB avg = 0.016 KB total ukey num = 0.070196846 Billion total entry num = 0.057004575 Billion write speed all = 0.274253402 MB/sec (with seq num) write speed all = 0.262876256 MB/sec (without seq num) 2024/11/23-20:03:15.828411 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278434: 925868 keys, 35852514 bytes, temperature: kWarm 2024/11/23-20:03:15.828481 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363395828447, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278434, "file_size": 35852514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4098574454, "largest_seqno": 4232676158, "table_properties": {"data_size": 29879280, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924304, "raw_average_key_size": 28, "raw_value_size": 108293536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925868, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363392, "oldest_key_time": 0, "file_creation_time": 1732363392, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278434, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:15.873844 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst) /dev/shm/ToplingTemp/Topling-174028-dV2fJl 2024/11/23-20:03:16.931743 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst): first pass time = 1.06's, 126.876'MB/sec 2024/11/23-20:03:18.423146 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst): old prealloc_size = 147639500, real_size = 32954592 2024/11/23-20:03:18.423877 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930389 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.26 avg-zval = 33.26 2024/11/23-20:03:18.424742 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:19.343388 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278435.sst): second pass time = 1.48's, 72.943'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2517.083'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.16's, 237.642'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34096.584'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.943'MB/sec zip pipeline throughput = 72.943'MB/sec entries = 930389 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 33.26 usrkeys = 930389 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930389 bytes = 3140080 } seq expand size = 6512723 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4461 value = 3.4960 dict = 1.00 all = 3.6327 } Zip/UnZip{ index = 0.0957 value = 0.2860 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 9.536814 GB avg = 0.165 KB (by entry num) total key len = 1.596971 GB avg = 0.028 KB total ukey len = 1.109414 GB avg = 0.016 KB total ukey num = 0.071127235 Billion total entry num = 0.057934964 Billion write speed all = 0.277739654 MB/sec (with seq num) write speed all = 0.266177846 MB/sec (without seq num) 2024/11/23-20:03:19.390480 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278435: 930389 keys, 36946706 bytes, temperature: kWarm 2024/11/23-20:03:19.390544 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363399390513, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278435, "file_size": 36946706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4129386720, "largest_seqno": 4232676442, "table_properties": {"data_size": 30940496, "index_size": 1781312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050892, "raw_average_key_size": 28, "raw_value_size": 108166934, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363395, "oldest_key_time": 0, "file_creation_time": 1732363395, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278435, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:19.425239 139957223556672 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst) /dev/shm/ToplingTemp/Topling-174028-LOXceo 2024/11/23-20:03:19.568454 139957223556672 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst): first pass time = 0.14's, 235.099'MB/sec 2024/11/23-20:03:20.120700 139957223556672 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst): old prealloc_size = 147639500, real_size = 9757456 2024/11/23-20:03:20.120939 139957223556672 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 233238 prefix = 4 raw-key = 0.0037 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0271 GB zip-val = 0.0092 GB avg-val = 116.31 avg-zval = 39.65 2024/11/23-20:03:20.121238 139957223556672 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.034 GB 2024/11/23-20:03:20.397624 139957223556672 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00/278436.sst): second pass time = 0.55's, 49.193'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 38.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2095.746'MB/sec (index lex order gen) rebuild zvType time = 0.24's, 0.245'MB/sec write SST data time = 0.04's, 276.444'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.279'MB, throughput = 9337.349'MB/sec zip my value time = 0.55's, unzip length = 0.027'GB zip my value throughput = 49.193'MB/sec zip pipeline throughput = 125.031'MB/sec entries = 233238 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.31 avg-zval = 39.65 usrkeys = 233238 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.31 avg-zval = 39.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 233238 bytes = 816352 } seq expand size = 1632666 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0047 GB value = 0.0271 GB dict = 0.28 MB all = 0.0337 GB } __ZipSize{ index = 0.0005 GB value = 0.0092 GB dict = 0.28 MB all = 0.0108 GB } UnZip/Zip{ index = 10.3474 value = 2.9333 dict = 1.00 all = 3.1174 } Zip/UnZip{ index = 0.0966 value = 0.3409 dict = 1.00 all = 0.3208 } ---------------------------- total value len = 9.565574 GB avg = 0.164 KB (by entry num) total key len = 1.603502 GB avg = 0.028 KB total ukey len = 1.114079 GB avg = 0.016 KB total ukey num = 0.071360473 Billion total entry num = 0.058168202 Billion write speed all = 0.278612671 MB/sec (with seq num) write speed all = 0.267004622 MB/sec (without seq num) 2024/11/23-20:03:20.417089 139957223556672 [compaction_job.cc:1948] [default] [JOB 34771] Generated table #278436: 233238 keys, 10796867 bytes, temperature: kWarm 2024/11/23-20:03:20.417158 139957223556672 EVENT_LOG_v1 {"time_micros": 1732363400417124, "cf_name": "default", "job": 34771, "event": "table_file_creation", "file_number": 278436, "file_size": 10796867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042106349, "largest_seqno": 4232676390, "table_properties": {"data_size": 9248128, "index_size": 450816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6530664, "raw_average_key_size": 28, "raw_value_size": 27127144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 233238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363399, "oldest_key_time": 0, "file_creation_time": 1732363399, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278436, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:20.431701 139957223556672 [compaction_job.cc:703] job-34771: subcompact[0], size: 0.231074 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34771/att-00 [278430,278431,278432,278433,278434,278435,278436] 2024/11/23-20:03:20.480430 139957223556672 (Original Log Time 2024/11/23-20:02:55.261529) EVENT_LOG_v1 {"time_micros": 1732363375261467, "job": 34771, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278407], "files_L3": [278378, 278379, 278380, 278336, 278337, 278338, 278419], "score": 0.997754, "input_data_size": 256320799, "oldest_snapshot_seqno": 4232747101} 2024/11/23-20:03:20.480434 139957223556672 (Original Log Time 2024/11/23-20:03:20.479700) [compaction_job.cc:2022] [default] [JOB 34771] Compacted 1@2 + 7@3 files to L3 => 231073821 bytes 2024/11/23-20:03:20.480436 139957223556672 (Original Log Time 2024/11/23-20:03:20.480367) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 7 0 0 0] max score 0.01, MB/sec: 10.2 rd, 9.2 wr, level 3, files in(1, 7) out(7 +0 blob) MB in(27.1, 217.4 +0.0 blob) out(220.4 +0.0 blob), read-write-amplify(17.2) write-amplify(8.1) OK, records in: 5872099, records dropped: 62061 output_compression: Snappy 2024/11/23-20:03:20.480442 139957223556672 (Original Log Time 2024/11/23-20:03:20.480409) EVENT_LOG_v1 {"time_micros": 1732363400480391, "cf": "default", "job": 34771, "event": "compaction_finished", "compaction_time_micros": 25170183, "compaction_time_cpu_micros": 10582495, "output_level": 3, "num_output_files": 7, "total_output_size": 231073821, "num_input_records": 5872099, "num_output_records": 5810038, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 177933558, "file_range_sync_nanos": 0, "file_fsync_nanos": 6260, "file_prepare_write_nanos": 17035, "lsm_state": [0, 0, 0, 7, 0, 0, 0]} 2024/11/23-20:03:20.614502 139957223556672 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732363375,"executesMs":25348,"compactionJobId":34771,"attempt":0,"compactionInputRawBytes":847777270,"compactionInputZipBytes":256320799,"compactionOutputRawBytes":0,"compactionOutputZipBytes":231073821,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7a5d1c26c6c88b8eb957f50f65726b93-e950a7156f322d34-00"}}, response = 2024/11/23-20:03:20.614581 139957223556672 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34771/att-00: olev 3, work 25.348 s, result 7.664 ms, install 0.759 ms, input{raw 808.503 MiB zip 244.447 MiB}