2024/11/22-19:40:51.465976 140024751826496 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00: fork to child time = 0.034610 sec 2024/11/22-19:40:51.466967 140024751826496 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00 2024/11/22-19:40:51.475562 140024751826496 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00 2024/11/22-19:40:51.476979 140024751826496 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-19:40:51.477343 140024751826496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-19:40:51.521525 140024751826496 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09135/att-00/MANIFEST-206347 2024/11/22-19:40:51.537129 140024751826496 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09135/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 223868, last_sequence is 3511133678, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 223710 2024/11/22-19:40:51.537140 140024751826496 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-19:40:51.537264 140024751826496 [version_set.cc:5721] Creating manifest 223868 2024/11/22-19:40:51.574317 140024751826496 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-19:40:51.606507 140024751826496 [compaction_job.cc:2437] [default] [JOB 9135] Compacting 1@2 + 12@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/22-19:40:51.606531 140024751826496 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6917 Base level 2, inputs: [223848(27MB)], [223785(35MB) 223786(36MB) 223787(36MB) 223788(36MB) 223789(37MB) 223790(37MB) 223791(35MB) 223792(36MB) 223793(35MB) 223794(35MB) 223795(35MB) 223863(34MB)] 2024/11/22-19:40:51.615091 140024751826496 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:40:51.615131 140024751826496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:40:51.615152 140024751826496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:40:51.615157 140024751826496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:40:51.615168 140024751826496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:40:52.082628 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst) /dev/shm/ToplingTemp/Topling-263487-FrWk5Q 2024/11/22-19:40:52.699795 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst): first pass time = 0.62's, 217.491'MB/sec 2024/11/22-19:40:56.438171 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst): old prealloc_size = 147639500, real_size = 33547488 2024/11/22-19:40:56.438748 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924163 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.23 avg-zval = 34.15 2024/11/22-19:40:56.439529 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:40:57.584888 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223869.sst): second pass time = 3.73's, 29.037'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 32.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3131.374'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.24's, 158.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 28692.210'MB/sec zip my value time = 3.73's, unzip length = 0.108'GB zip my value throughput = 29.037'MB/sec zip pipeline throughput = 101.663'MB/sec entries = 924163 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.23 avg-zval = 34.15 usrkeys = 924163 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.23 avg-zval = 34.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924163 bytes = 3119072 } seq expand size = 6469141 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5137 value = 3.4331 dict = 1.00 all = 3.5769 } Zip/UnZip{ index = 0.0951 value = 0.2913 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 0.114810 GB avg = 0.124 KB (by entry num) total key len = 0.025877 GB avg = 0.028 KB total ukey len = 0.018483 GB avg = 0.020 KB total ukey num = 0.000924163 Billion total entry num = 0.000924163 Billion write speed all = 0.003615284 MB/sec (with seq num) write speed all = 0.003425296 MB/sec (without seq num) 2024/11/22-19:40:57.619994 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223869: 924163 keys, 37523394 bytes, temperature: kWarm 2024/11/22-19:40:57.620084 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275657620030, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223869, "file_size": 37523394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038008, "largest_seqno": 3511052197, "table_properties": {"data_size": 31558224, "index_size": 1758016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876564, "raw_average_key_size": 28, "raw_value_size": 108341242, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275652, "oldest_key_time": 0, "file_creation_time": 1732275652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:40:57.656771 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst) /dev/shm/ToplingTemp/Topling-263487-HgNjOO 2024/11/22-19:40:58.862367 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst): first pass time = 1.21's, 111.333'MB/sec 2024/11/22-19:41:00.694081 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst): old prealloc_size = 147639500, real_size = 34128976 2024/11/22-19:41:00.694659 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926129 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.92 avg-zval = 34.69 2024/11/22-19:41:00.695442 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:01.622333 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223870.sst): second pass time = 1.83's, 59.138'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3182.756'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.14's, 273.516'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36555.274'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.138'MB/sec zip pipeline throughput = 59.138'MB/sec entries = 926129 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.92 avg-zval = 34.69 usrkeys = 926129 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.92 avg-zval = 34.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926129 bytes = 3125696 } seq expand size = 6482903 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4429 value = 3.3709 dict = 1.00 all = 3.5219 } Zip/UnZip{ index = 0.0958 value = 0.2967 dict = 1.00 all = 0.2839 } ---------------------------- total value len = 0.229579 GB avg = 0.124 KB (by entry num) total key len = 0.051808 GB avg = 0.028 KB total ukey len = 0.037006 GB avg = 0.020 KB total ukey num = 0.001850292 Billion total entry num = 0.001850292 Billion write speed all = 0.007230172 MB/sec (with seq num) write speed all = 0.006849830 MB/sec (without seq num) 2024/11/22-19:41:01.654575 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223870: 926129 keys, 38109326 bytes, temperature: kWarm 2024/11/22-19:41:01.654644 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275661654611, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223870, "file_size": 38109326, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3511053744, "table_properties": {"data_size": 32123584, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931612, "raw_average_key_size": 28, "raw_value_size": 108286182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926129, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275657, "oldest_key_time": 0, "file_creation_time": 1732275657, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:01.693281 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst) /dev/shm/ToplingTemp/Topling-263487-U1AigR 2024/11/22-19:41:02.934345 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst): first pass time = 1.24's, 108.151'MB/sec 2024/11/22-19:41:04.839894 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst): old prealloc_size = 147639500, real_size = 34296416 2024/11/22-19:41:04.840445 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926951 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0323 GB avg-val = 116.79 avg-zval = 34.85 2024/11/22-19:41:04.841223 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:05.751740 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223871.sst): second pass time = 1.90's, 56.846'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.486'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 300.946'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34874.092'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 56.846'MB/sec zip pipeline throughput = 56.846'MB/sec entries = 926951 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 34.85 usrkeys = 926951 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 34.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926951 bytes = 3128480 } seq expand size = 6488657 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5140 value = 3.3517 dict = 1.00 all = 3.5063 } Zip/UnZip{ index = 0.0951 value = 0.2984 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 0.344331 GB avg = 0.124 KB (by entry num) total key len = 0.077763 GB avg = 0.028 KB total ukey len = 0.055545 GB avg = 0.020 KB total ukey num = 0.002777243 Billion total entry num = 0.002777243 Billion write speed all = 0.010844431 MB/sec (with seq num) write speed all = 0.010273608 MB/sec (without seq num) 2024/11/22-19:41:05.785297 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223871: 926951 keys, 38278874 bytes, temperature: kWarm 2024/11/22-19:41:05.785362 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275665785330, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223871, "file_size": 38278874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3511052484, "table_properties": {"data_size": 32301200, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954628, "raw_average_key_size": 28, "raw_value_size": 108263118, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926951, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275661, "oldest_key_time": 0, "file_creation_time": 1732275661, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:05.845592 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst) /dev/shm/ToplingTemp/Topling-263487-a06UHQ 2024/11/22-19:41:07.160652 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst): first pass time = 1.32's, 102.066'MB/sec 2024/11/22-19:41:08.900483 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst): old prealloc_size = 147639500, real_size = 34097312 2024/11/22-19:41:08.901160 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926712 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.83 avg-zval = 34.64 2024/11/22-19:41:08.901928 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:09.887953 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223872.sst): second pass time = 1.74's, 62.253'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 38.564'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2734.117'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.19's, 198.152'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26478.951'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.253'MB/sec zip pipeline throughput = 62.253'MB/sec entries = 926712 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 34.64 usrkeys = 926712 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 34.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926712 bytes = 3127664 } seq expand size = 6486984 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5129 value = 3.3726 dict = 1.00 all = 3.5247 } Zip/UnZip{ index = 0.0951 value = 0.2965 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 0.459088 GB avg = 0.124 KB (by entry num) total key len = 0.103711 GB avg = 0.028 KB total ukey len = 0.074079 GB avg = 0.020 KB total ukey num = 0.003703955 Billion total entry num = 0.003703955 Billion write speed all = 0.014457879 MB/sec (with seq num) write speed all = 0.013696664 MB/sec (without seq num) 2024/11/22-19:41:09.935229 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223872: 926712 keys, 38079662 bytes, temperature: kWarm 2024/11/22-19:41:09.935311 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275669935267, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223872, "file_size": 38079662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3511053235, "table_properties": {"data_size": 32102416, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947936, "raw_average_key_size": 28, "raw_value_size": 108269876, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926712, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275665, "oldest_key_time": 0, "file_creation_time": 1732275665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:10.003927 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst) /dev/shm/ToplingTemp/Topling-263487-E6eekR 2024/11/22-19:41:11.403542 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst): first pass time = 1.40's, 95.900'MB/sec 2024/11/22-19:41:13.005127 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst): old prealloc_size = 147639500, real_size = 35178144 2024/11/22-19:41:13.005815 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929020 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.47 avg-zval = 35.71 2024/11/22-19:41:13.006662 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:14.295767 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223873.sst): second pass time = 1.60's, 67.655'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 30.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2663.745'MB/sec (index lex order gen) rebuild zvType time = 1.06's, 0.218'MB/sec write SST data time = 0.23's, 175.051'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26446.960'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.655'MB/sec zip pipeline throughput = 67.655'MB/sec entries = 929020 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 35.71 usrkeys = 929020 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 35.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929020 bytes = 3367712 } seq expand size = 6503140 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5135 value = 3.2613 dict = 1.00 all = 3.4063 } Zip/UnZip{ index = 0.0951 value = 0.3066 dict = 1.00 all = 0.2936 } ---------------------------- total value len = 0.573796 GB avg = 0.124 KB (by entry num) total key len = 0.129723 GB avg = 0.028 KB total ukey len = 0.092660 GB avg = 0.020 KB total ukey num = 0.004632975 Billion total entry num = 0.004632975 Billion write speed all = 0.018070848 MB/sec (with seq num) write speed all = 0.017118815 MB/sec (without seq num) 2024/11/22-19:41:14.330061 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223873: 929020 keys, 39402362 bytes, temperature: kWarm 2024/11/22-19:41:14.330124 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275674330091, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223873, "file_size": 39402362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3511054125, "table_properties": {"data_size": 33178384, "index_size": 1767296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012560, "raw_average_key_size": 28, "raw_value_size": 108205248, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929020, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275669, "oldest_key_time": 0, "file_creation_time": 1732275669, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:14.366145 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst) /dev/shm/ToplingTemp/Topling-263487-yFOzgQ 2024/11/22-19:41:15.243550 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst): first pass time = 0.88's, 152.978'MB/sec 2024/11/22-19:41:17.374261 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst): old prealloc_size = 147639500, real_size = 37152528 2024/11/22-19:41:17.374812 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925724 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.99 avg-zval = 37.98 2024/11/22-19:41:17.375183 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:18.197749 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223874.sst): second pass time = 2.13's, 50.844'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.655'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.706'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.394'MB/sec write SST data time = 0.24's, 166.546'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25259.159'MB/sec zip my value time = 2.13's, unzip length = 0.108'GB zip my value throughput = 50.844'MB/sec zip pipeline throughput = 112.643'MB/sec entries = 925724 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 37.98 usrkeys = 925724 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.51 avg-zval = 37.98 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 334193 bytes = 1086144 } seq expand size = 2339351 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5127 value = 3.0802 dict = 1.00 all = 3.4203 } Zip/UnZip{ index = 0.0951 value = 0.3247 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 0.684433 GB avg = 0.123 KB (by entry num) total key len = 0.155644 GB avg = 0.028 KB total ukey len = 0.111174 GB avg = 0.020 KB total ukey num = 0.005558699 Billion total entry num = 0.005558699 Billion write speed all = 0.021576338 MB/sec (with seq num) write speed all = 0.020434192 MB/sec (without seq num) 2024/11/22-19:41:18.238049 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223874: 925724 keys, 39241961 bytes, temperature: kWarm 2024/11/22-19:41:18.238131 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275678238089, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223874, "file_size": 39241961, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35159744, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920272, "raw_average_key_size": 28, "raw_value_size": 108297508, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275674, "oldest_key_time": 0, "file_creation_time": 1732275674, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223874, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:18.308014 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst) /dev/shm/ToplingTemp/Topling-263487-FlD4UO 2024/11/22-19:41:19.207605 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst): first pass time = 0.90's, 149.208'MB/sec 2024/11/22-19:41:21.104394 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst): old prealloc_size = 147639500, real_size = 36469936 2024/11/22-19:41:21.104979 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927056 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.20 2024/11/22-19:41:21.105022 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:21.746987 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223875.sst): second pass time = 1.90's, 57.102'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.091'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.219'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.450'MB/sec write SST data time = 0.13's, 293.003'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33258.682'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 57.102'MB/sec zip pipeline throughput = 59.316'MB/sec entries = 927056 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.20 usrkeys = 927056 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5821 value = 3.1393 dict = 1.00 all = 3.5956 } Zip/UnZip{ index = 0.0945 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.792694 GB avg = 0.122 KB (by entry num) total key len = 0.181601 GB avg = 0.028 KB total ukey len = 0.129715 GB avg = 0.020 KB total ukey num = 0.006485755 Billion total entry num = 0.006485755 Billion write speed all = 0.025021275 MB/sec (with seq num) write speed all = 0.023688767 MB/sec (without seq num) 2024/11/22-19:41:21.780052 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223875: 927056 keys, 37327924 bytes, temperature: kWarm 2024/11/22-19:41:21.780117 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275681780083, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223875, "file_size": 37327924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34485856, "index_size": 1752128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957568, "raw_average_key_size": 28, "raw_value_size": 108260192, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927056, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275678, "oldest_key_time": 0, "file_creation_time": 1732275678, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223875, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:21.816404 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst) /dev/shm/ToplingTemp/Topling-263487-IEJMCP 2024/11/22-19:41:22.637254 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst): first pass time = 0.82's, 163.518'MB/sec 2024/11/22-19:41:24.362305 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst): old prealloc_size = 147639500, real_size = 36592640 2024/11/22-19:41:24.362896 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924735 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.14 avg-zval = 37.42 2024/11/22-19:41:24.363196 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:25.256240 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223876.sst): second pass time = 1.72's, 62.848'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3087.088'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.24's, 162.291'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28644.337'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.848'MB/sec zip pipeline throughput = 87.201'MB/sec entries = 924735 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 37.42 usrkeys = 924735 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.15 avg-zval = 37.42 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 265551 bytes = 863056 } seq expand size = 1858857 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5118 value = 3.1306 dict = 1.00 all = 3.4898 } Zip/UnZip{ index = 0.0951 value = 0.3194 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 0.902878 GB avg = 0.122 KB (by entry num) total key len = 0.207494 GB avg = 0.028 KB total ukey len = 0.148210 GB avg = 0.020 KB total ukey num = 0.007410490 Billion total entry num = 0.007410490 Billion write speed all = 0.028513348 MB/sec (with seq num) write speed all = 0.026990990 MB/sec (without seq num) 2024/11/22-19:41:25.292402 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223876: 924735 keys, 38459747 bytes, temperature: kWarm 2024/11/22-19:41:25.292485 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275685292442, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223876, "file_size": 38459747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3328170471, "table_properties": {"data_size": 34601840, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892580, "raw_average_key_size": 28, "raw_value_size": 108325250, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924735, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275681, "oldest_key_time": 0, "file_creation_time": 1732275681, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223876, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:25.354442 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst) /dev/shm/ToplingTemp/Topling-263487-tv1cvP 2024/11/22-19:41:26.255789 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst): first pass time = 0.90's, 148.917'MB/sec 2024/11/22-19:41:27.729197 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst): old prealloc_size = 147639500, real_size = 32835360 2024/11/22-19:41:27.729714 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925367 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.04 avg-zval = 33.33 2024/11/22-19:41:27.730495 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:28.648102 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223877.sst): second pass time = 1.47's, 73.542'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.189'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3571.533'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 289.199'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27923.264'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.542'MB/sec zip pipeline throughput = 73.542'MB/sec entries = 925367 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 33.33 usrkeys = 925367 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925367 bytes = 3238800 } seq expand size = 6477569 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5125 value = 3.5115 dict = 1.00 all = 3.6344 } Zip/UnZip{ index = 0.0951 value = 0.2848 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 1.017663 GB avg = 0.122 KB (by entry num) total key len = 0.233404 GB avg = 0.028 KB total ukey len = 0.166717 GB avg = 0.020 KB total ukey num = 0.008335857 Billion total entry num = 0.008335857 Billion write speed all = 0.032123482 MB/sec (with seq num) write speed all = 0.030411172 MB/sec (without seq num) 2024/11/22-19:41:28.684137 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223877: 925367 keys, 36929398 bytes, temperature: kWarm 2024/11/22-19:41:28.684207 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275688684171, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223877, "file_size": 36929398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3328170472, "largest_seqno": 3511053329, "table_properties": {"data_size": 30843344, "index_size": 1760512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910276, "raw_average_key_size": 28, "raw_value_size": 108307542, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925367, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275685, "oldest_key_time": 0, "file_creation_time": 1732275685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223877, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:28.720892 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst) /dev/shm/ToplingTemp/Topling-263487-4ik7FR 2024/11/22-19:41:29.539950 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst): first pass time = 0.82's, 163.877'MB/sec 2024/11/22-19:41:31.298141 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst): old prealloc_size = 147639500, real_size = 33979200 2024/11/22-19:41:31.298682 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924366 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.20 avg-zval = 34.59 2024/11/22-19:41:31.300348 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:32.344084 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223878.sst): second pass time = 1.76's, 61.641'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 36.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3414.718'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.24's, 161.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30924.018'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.641'MB/sec zip pipeline throughput = 119.641'MB/sec entries = 924366 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.20 avg-zval = 34.59 usrkeys = 924366 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.20 avg-zval = 34.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924366 bytes = 3235296 } seq expand size = 6470562 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4423 value = 3.3879 dict = 1.00 all = 3.5255 } Zip/UnZip{ index = 0.0958 value = 0.2952 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 1.132469 GB avg = 0.122 KB (by entry num) total key len = 0.259286 GB avg = 0.028 KB total ukey len = 0.185204 GB avg = 0.020 KB total ukey num = 0.009260223 Billion total entry num = 0.009260223 Billion write speed all = 0.035732528 MB/sec (with seq num) write speed all = 0.033830520 MB/sec (without seq num) 2024/11/22-19:41:32.384136 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223878: 924366 keys, 38071048 bytes, temperature: kWarm 2024/11/22-19:41:32.384219 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275692384176, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223878, "file_size": 38071048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3511054066, "table_properties": {"data_size": 31977520, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882248, "raw_average_key_size": 28, "raw_value_size": 108335544, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924366, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275688, "oldest_key_time": 0, "file_creation_time": 1732275688, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223878, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:32.457837 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst) /dev/shm/ToplingTemp/Topling-263487-k3ekjS 2024/11/22-19:41:33.400458 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst): first pass time = 0.94's, 142.396'MB/sec 2024/11/22-19:41:34.885338 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst): old prealloc_size = 147639500, real_size = 33646720 2024/11/22-19:41:34.885911 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923435 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.35 avg-zval = 34.28 2024/11/22-19:41:34.888734 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:36.050247 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223879.sst): second pass time = 1.48's, 73.037'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 32.296'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.949'MB/sec (index lex order gen) rebuild zvType time = 1.03's, 0.225'MB/sec write SST data time = 0.14's, 271.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35476.936'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.037'MB/sec zip pipeline throughput = 73.037'MB/sec entries = 923435 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 34.28 usrkeys = 923435 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 34.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923435 bytes = 3116608 } seq expand size = 6464045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5112 value = 3.4228 dict = 1.00 all = 3.5676 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 1.247295 GB avg = 0.122 KB (by entry num) total key len = 0.285142 GB avg = 0.028 KB total ukey len = 0.203673 GB avg = 0.020 KB total ukey num = 0.010183658 Billion total entry num = 0.010183658 Billion write speed all = 0.039340712 MB/sec (with seq num) write speed all = 0.037249234 MB/sec (without seq num) 2024/11/22-19:41:36.087684 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223879: 923435 keys, 37621218 bytes, temperature: kWarm 2024/11/22-19:41:36.087756 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275696087724, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223879, "file_size": 37621218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3511053359, "table_properties": {"data_size": 31658608, "index_size": 1757056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856180, "raw_average_key_size": 28, "raw_value_size": 108361622, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275692, "oldest_key_time": 0, "file_creation_time": 1732275692, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223879, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:36.123925 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst) /dev/shm/ToplingTemp/Topling-263487-VySRvS 2024/11/22-19:41:37.156517 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst): first pass time = 1.03's, 129.987'MB/sec 2024/11/22-19:41:38.681123 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst): old prealloc_size = 147639500, real_size = 33120352 2024/11/22-19:41:38.681761 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924247 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.22 avg-zval = 33.68 2024/11/22-19:41:38.682595 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:41:39.758263 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223880.sst): second pass time = 1.52's, 71.103'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 34.547'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2798.413'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.251'MB/sec write SST data time = 0.16's, 239.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31051.206'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.103'MB/sec zip pipeline throughput = 71.103'MB/sec entries = 924247 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 33.68 usrkeys = 924247 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924247 bytes = 3234880 } seq expand size = 6469729 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5147 value = 3.4801 dict = 1.00 all = 3.6069 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 1.362103 GB avg = 0.123 KB (by entry num) total key len = 0.311021 GB avg = 0.028 KB total ukey len = 0.222158 GB avg = 0.020 KB total ukey num = 0.011107905 Billion total entry num = 0.011107905 Billion write speed all = 0.042948354 MB/sec (with seq num) write speed all = 0.040667275 MB/sec (without seq num) 2024/11/22-19:41:39.793475 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223880: 924247 keys, 37211824 bytes, temperature: kWarm 2024/11/22-19:41:39.793536 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275699793504, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223880, "file_size": 37211824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3511053374, "table_properties": {"data_size": 31131088, "index_size": 1758016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25878916, "raw_average_key_size": 28, "raw_value_size": 108338930, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924247, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275696, "oldest_key_time": 0, "file_creation_time": 1732275696, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223880, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:39.832720 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst) /dev/shm/ToplingTemp/Topling-263487-tmBEpQ 2024/11/22-19:41:39.920722 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst): first pass time = 0.09's, 235.024'MB/sec 2024/11/22-19:41:40.106849 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst): old prealloc_size = 147639500, real_size = 5874112 2024/11/22-19:41:40.106988 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 144684 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0166 GB zip-val = 0.0056 GB avg-val = 114.88 avg-zval = 38.44 2024/11/22-19:41:40.107143 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/22-19:41:40.238580 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00/223881.sst): second pass time = 0.19's, 89.617'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 49.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2417.946'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.325'MB/sec write SST data time = 0.02's, 320.713'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.175'MB, throughput = 6098.742'MB/sec zip my value time = 0.19's, unzip length = 0.017'GB zip my value throughput = 89.617'MB/sec zip pipeline throughput = 89.617'MB/sec entries = 144684 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.88 avg-zval = 38.44 usrkeys = 144684 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.88 avg-zval = 38.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 144684 bytes = 488320 } seq expand size = 1012788 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0166 GB dict = 0.17 MB all = 0.0207 GB } __ZipSize{ index = 0.0003 GB value = 0.0056 GB dict = 0.17 MB all = 0.0065 GB } UnZip/Zip{ index = 10.4661 value = 2.9888 dict = 1.00 all = 3.1790 } Zip/UnZip{ index = 0.0955 value = 0.3346 dict = 1.00 all = 0.3146 } ---------------------------- total value len = 1.379737 GB avg = 0.123 KB (by entry num) total key len = 0.315072 GB avg = 0.028 KB total ukey len = 0.225052 GB avg = 0.020 KB total ukey num = 0.011252589 Billion total entry num = 0.011252589 Billion write speed all = 0.043504471 MB/sec (with seq num) write speed all = 0.041193708 MB/sec (without seq num) 2024/11/22-19:41:40.249911 140024751826496 [compaction_job.cc:1948] [default] [JOB 9135] Generated table #223881: 144684 keys, 6502753 bytes, temperature: kWarm 2024/11/22-19:41:40.249973 140024751826496 EVENT_LOG_v1 {"time_micros": 1732275700249939, "cf_name": "default", "job": 9135, "event": "table_file_creation", "file_number": 223881, "file_size": 6502753, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3509575995, "table_properties": {"data_size": 5561264, "index_size": 276480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4051152, "raw_average_key_size": 28, "raw_value_size": 16621396, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 144684, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275699, "oldest_key_time": 0, "file_creation_time": 1732275699, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223881, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:41:40.261171 140024751826496 [compaction_job.cc:703] job-09135: subcompact[0], size: 0.462759 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09135/att-00 [223869,223870,223871,223872,223873,223874,223875,223876,223877,223878,223879,223880,223881] 2024/11/22-19:41:40.348864 140024751826496 (Original Log Time 2024/11/22-19:40:51.606607) EVENT_LOG_v1 {"time_micros": 1732275651606551, "job": 9135, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [223848], "files_L3": [223785, 223786, 223787, 223788, 223789, 223790, 223791, 223792, 223793, 223794, 223795, 223863], "score": 0.984428, "input_data_size": 483686006, "oldest_snapshot_seqno": 3511133668} 2024/11/22-19:41:40.348867 140024751826496 (Original Log Time 2024/11/22-19:41:40.348147) [compaction_job.cc:2022] [default] [JOB 9135] Compacted 1@2 + 12@3 files to L3 => 462759491 bytes 2024/11/22-19:41:40.348869 140024751826496 (Original Log Time 2024/11/22-19:41:40.348795) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 9.9 rd, 9.5 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 434.2 +0.0 blob) out(441.3 +0.0 blob), read-write-amplify(33.3) write-amplify(16.3) OK, records in: 11343019, records dropped: 90430 output_compression: Snappy 2024/11/22-19:41:40.348875 140024751826496 (Original Log Time 2024/11/22-19:41:40.348843) EVENT_LOG_v1 {"time_micros": 1732275700348819, "cf": "default", "job": 9135, "event": "compaction_finished", "compaction_time_micros": 48654569, "compaction_time_cpu_micros": 19808928, "output_level": 3, "num_output_files": 13, "total_output_size": 462759491, "num_input_records": 11343019, "num_output_records": 11252589, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 330508519, "file_range_sync_nanos": 0, "file_fsync_nanos": 13407, "file_prepare_write_nanos": 32622, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-19:41:40.491997 140024751826496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732275651,"executesMs":48876,"compactionJobId":9135,"attempt":0,"compactionInputRawBytes":1644127202,"compactionInputZipBytes":483686006,"compactionOutputRawBytes":0,"compactionOutputZipBytes":462759491,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-ba1944af7a2d3a716b76f6a7d63ed3af-434a4ea537e3c14c-00"}}, response = 2024/11/22-19:41:40.492080 140024751826496 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09135/att-00: olev 3, work 48.877 s, result 4.926 ms, install 0.749 ms, input{raw 1.531 GiB zip 461.279 MiB}