2024/11/22-16:45:45.123369 140024844146240 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00: fork to child time = 0.011201 sec 2024/11/22-16:45:45.124126 140024844146240 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00 2024/11/22-16:45:45.130654 140024844146240 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00 2024/11/22-16:45:45.131764 140024844146240 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-16:45:45.132012 140024844146240 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-16:45:45.180750 140024844146240 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04994/att-00/MANIFEST-206347 2024/11/22-16:45:45.202189 140024844146240 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04994/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 215470, last_sequence is 3392491010, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 215269 2024/11/22-16:45:45.202195 140024844146240 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-16:45:45.202320 140024844146240 [version_set.cc:5721] Creating manifest 215470 2024/11/22-16:45:45.236472 140024844146240 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-16:45:45.261361 140024844146240 [compaction_job.cc:2437] [default] [JOB 4994] Compacting 1@2 + 12@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/22-16:45:45.261377 140024844146240 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 3881 Base level 2, inputs: [215422(27MB)], [215467(38MB) 215468(7470KB) 215372(34MB) 215373(35MB) 215374(35MB) 215375(36MB) 215376(37MB) 215377(37MB) 215378(12MB) 215339(35MB) 215340(36MB) 215433(35MB)] 2024/11/22-16:45:45.267090 140024844146240 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:45:45.267119 140024844146240 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:45:45.267137 140024844146240 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-16:45:45.267142 140024844146240 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:45:45.267152 140024844146240 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-16:45:45.831643 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst) /dev/shm/ToplingTemp/Topling-205010-CvHjO7 2024/11/22-16:45:46.364387 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst): first pass time = 0.53's, 251.957'MB/sec 2024/11/22-16:45:47.729402 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst): old prealloc_size = 147639500, real_size = 36119744 2024/11/22-16:45:47.729981 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930061 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0341 GB avg-val = 116.31 avg-zval = 36.67 2024/11/22-16:45:47.730741 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:45:48.568605 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215471.sst): second pass time = 1.36's, 79.323'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.796'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3174.087'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 320.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27456.340'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.323'MB/sec zip pipeline throughput = 79.323'MB/sec entries = 930061 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.31 avg-zval = 36.67 usrkeys = 930061 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.31 avg-zval = 36.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930061 bytes = 3138976 } seq expand size = 6510427 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4436 value = 3.1718 dict = 1.00 all = 3.3461 } Zip/UnZip{ index = 0.0958 value = 0.3153 dict = 1.00 all = 0.2989 } ---------------------------- total value len = 0.114687 GB avg = 0.123 KB (by entry num) total key len = 0.026042 GB avg = 0.028 KB total ukey len = 0.018601 GB avg = 0.020 KB total ukey num = 0.000930061 Billion total entry num = 0.000930061 Billion write speed all = 0.004954266 MB/sec (with seq num) write speed all = 0.004692327 MB/sec (without seq num) 2024/11/22-16:45:48.607274 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215471: 930061 keys, 40111160 bytes, temperature: kWarm 2024/11/22-16:45:48.607358 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265148607306, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215471, "file_size": 40111160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3280787737, "largest_seqno": 3391789750, "table_properties": {"data_size": 34105904, "index_size": 1781120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26041708, "raw_average_key_size": 28, "raw_value_size": 108176126, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930061, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265145, "oldest_key_time": 0, "file_creation_time": 1732265145, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215471, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:45:48.642229 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst) /dev/shm/ToplingTemp/Topling-205010-3QhPh4 2024/11/22-16:45:49.570603 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst): first pass time = 0.93's, 144.578'MB/sec 2024/11/22-16:45:50.927687 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst): old prealloc_size = 147639500, real_size = 35190336 2024/11/22-16:45:50.928295 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083847 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.83 avg-zval = 30.30 2024/11/22-16:45:50.928586 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:45:51.603869 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215472.sst): second pass time = 1.36's, 76.599'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 54.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3539.331'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.505'MB/sec write SST data time = 0.14's, 265.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 33185.388'MB/sec zip my value time = 1.36's, unzip length = 0.104'GB zip my value throughput = 76.599'MB/sec zip pipeline throughput = 76.599'MB/sec entries = 1083847 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.83 avg-zval = 30.30 usrkeys = 1083847 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.27 avg-zval = 30.30 TagRS{ kinds = RS_Key0_TagN bytes = 169440 } TagArray{ size = 222939 bytes = 668832 } seq expand size = 1560573 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.04 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4473 value = 3.1625 dict = 1.00 all = 3.6467 } Zip/UnZip{ index = 0.0957 value = 0.3162 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 0.220117 GB avg = 0.109 KB (by entry num) total key len = 0.056389 GB avg = 0.028 KB total ukey len = 0.040278 GB avg = 0.020 KB total ukey num = 0.002013908 Billion total entry num = 0.002013908 Billion write speed all = 0.009733234 MB/sec (with seq num) write speed all = 0.009166106 MB/sec (without seq num) 2024/11/22-16:45:51.639765 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215472: 1083847 keys, 36804800 bytes, temperature: kWarm 2024/11/22-16:45:51.639831 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265151639799, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215472, "file_size": 36804800, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3391789181, "table_properties": {"data_size": 32844336, "index_size": 2074880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30347716, "raw_average_key_size": 28, "raw_value_size": 103870111, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083847, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265148, "oldest_key_time": 0, "file_creation_time": 1732265148, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:45:51.675149 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst) /dev/shm/ToplingTemp/Topling-205010-YutGj4 2024/11/22-16:45:52.516242 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst): first pass time = 0.84's, 159.581'MB/sec 2024/11/22-16:45:53.917703 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst): old prealloc_size = 147639500, real_size = 35963744 2024/11/22-16:45:53.918233 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923669 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0340 GB avg-val = 117.31 avg-zval = 36.78 2024/11/22-16:45:53.918633 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:45:54.682764 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215473.sst): second pass time = 1.40's, 77.351'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 50.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3457.101'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 305.813'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34448.928'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.351'MB/sec zip pipeline throughput = 77.351'MB/sec entries = 923669 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 36.78 usrkeys = 923669 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.31 avg-zval = 36.78 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 396174 bytes = 1337104 } seq expand size = 2773218 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5134 value = 3.1892 dict = 1.00 all = 3.5038 } Zip/UnZip{ index = 0.0951 value = 0.3136 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 0.331245 GB avg = 0.113 KB (by entry num) total key len = 0.082252 GB avg = 0.028 KB total ukey len = 0.058752 GB avg = 0.020 KB total ukey num = 0.002937577 Billion total entry num = 0.002937577 Billion write speed all = 0.014553838 MB/sec (with seq num) write speed all = 0.013726689 MB/sec (without seq num) 2024/11/22-16:45:54.719094 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215473: 923669 keys, 38306671 bytes, temperature: kWarm 2024/11/22-16:45:54.719155 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265154719124, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215473, "file_size": 38306671, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3349211583, "table_properties": {"data_size": 33975504, "index_size": 1757120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862732, "raw_average_key_size": 28, "raw_value_size": 108355018, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265151, "oldest_key_time": 0, "file_creation_time": 1732265151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:45:54.752065 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst) /dev/shm/ToplingTemp/Topling-205010-w8qtp7 2024/11/22-16:45:55.570299 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst): first pass time = 0.82's, 164.041'MB/sec 2024/11/22-16:45:56.910387 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst): old prealloc_size = 147639500, real_size = 33684640 2024/11/22-16:45:56.910937 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921387 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.67 avg-zval = 34.41 2024/11/22-16:45:56.911703 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:45:57.828446 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215474.sst): second pass time = 1.34's, 80.945'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.015'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.048'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.13's, 297.686'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 27131.149'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.945'MB/sec zip pipeline throughput = 80.945'MB/sec entries = 921387 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.67 avg-zval = 34.41 usrkeys = 921387 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.67 avg-zval = 34.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921387 bytes = 3109696 } seq expand size = 6449709 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5139 value = 3.4200 dict = 1.00 all = 3.5645 } Zip/UnZip{ index = 0.0951 value = 0.2924 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.446114 GB avg = 0.116 KB (by entry num) total key len = 0.108051 GB avg = 0.028 KB total ukey len = 0.077179 GB avg = 0.020 KB total ukey num = 0.003858964 Billion total entry num = 0.003858964 Billion write speed all = 0.019502741 MB/sec (with seq num) write speed all = 0.018416272 MB/sec (without seq num) 2024/11/22-16:45:57.861458 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215474: 921387 keys, 37653616 bytes, temperature: kWarm 2024/11/22-16:45:57.861518 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265157861486, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215474, "file_size": 37653616, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701073, "largest_seqno": 3391789292, "table_properties": {"data_size": 31701392, "index_size": 1752704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25798836, "raw_average_key_size": 28, "raw_value_size": 108418982, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921387, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265154, "oldest_key_time": 0, "file_creation_time": 1732265154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:45:57.895452 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst) /dev/shm/ToplingTemp/Topling-205010-Jxms24 2024/11/22-16:45:58.776596 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst): first pass time = 0.88's, 152.329'MB/sec 2024/11/22-16:46:00.111530 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst): old prealloc_size = 147639500, real_size = 33588688 2024/11/22-16:46:00.112070 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923554 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.33 avg-zval = 34.22 2024/11/22-16:46:00.112828 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:01.006088 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215475.sst): second pass time = 1.33's, 81.242'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.992'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3387.544'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 296.219'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32249.248'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.242'MB/sec zip pipeline throughput = 81.242'MB/sec entries = 923554 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 34.22 usrkeys = 923554 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 34.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923554 bytes = 3117008 } seq expand size = 6464878 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5114 value = 3.4290 dict = 1.00 all = 3.5731 } Zip/UnZip{ index = 0.0951 value = 0.2916 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 0.560937 GB avg = 0.117 KB (by entry num) total key len = 0.133911 GB avg = 0.028 KB total ukey len = 0.095650 GB avg = 0.020 KB total ukey num = 0.004782518 Billion total entry num = 0.004782518 Billion write speed all = 0.024451052 MB/sec (with seq num) write speed all = 0.023104713 MB/sec (without seq num) 2024/11/22-16:46:01.040508 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215475: 923554 keys, 37563360 bytes, temperature: kWarm 2024/11/22-16:46:01.040574 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265161040542, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215475, "file_size": 37563360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701084, "largest_seqno": 3391789889, "table_properties": {"data_size": 31600384, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859512, "raw_average_key_size": 28, "raw_value_size": 108358252, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923554, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265157, "oldest_key_time": 0, "file_creation_time": 1732265157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:01.077136 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst) /dev/shm/ToplingTemp/Topling-205010-NBV1n6 2024/11/22-16:46:01.955254 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst): first pass time = 0.88's, 152.854'MB/sec 2024/11/22-16:46:03.273607 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst): old prealloc_size = 147639500, real_size = 34414800 2024/11/22-16:46:03.274154 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923704 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0324 GB avg-val = 117.30 avg-zval = 35.08 2024/11/22-16:46:03.274903 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:04.174692 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215476.sst): second pass time = 1.32's, 82.262'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3312.038'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.298'MB/sec write SST data time = 0.13's, 305.654'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32031.247'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.262'MB/sec zip pipeline throughput = 82.262'MB/sec entries = 923704 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.30 avg-zval = 35.08 usrkeys = 923704 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.30 avg-zval = 35.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923704 bytes = 3117520 } seq expand size = 6465928 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3740 value = 3.3440 dict = 1.00 all = 3.4962 } Zip/UnZip{ index = 0.0964 value = 0.2990 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 0.675757 GB avg = 0.118 KB (by entry num) total key len = 0.159774 GB avg = 0.028 KB total ukey len = 0.114124 GB avg = 0.020 KB total ukey num = 0.005706222 Billion total entry num = 0.005706222 Billion write speed all = 0.029398307 MB/sec (with seq num) write speed all = 0.027792112 MB/sec (without seq num) 2024/11/22-16:46:04.208239 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215476: 923704 keys, 38389712 bytes, temperature: kWarm 2024/11/22-16:46:04.208301 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265164208269, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215476, "file_size": 38389712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3391789723, "table_properties": {"data_size": 32402880, "index_size": 1780800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25863712, "raw_average_key_size": 28, "raw_value_size": 108354152, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265161, "oldest_key_time": 0, "file_creation_time": 1732265161, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:04.243965 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst) /dev/shm/ToplingTemp/Topling-205010-ip7MV7 2024/11/22-16:46:05.125463 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst): first pass time = 0.88's, 152.269'MB/sec 2024/11/22-16:46:06.461570 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst): old prealloc_size = 147639500, real_size = 35612112 2024/11/22-16:46:06.462104 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927932 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.64 avg-zval = 36.22 2024/11/22-16:46:06.462767 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:07.309287 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215477.sst): second pass time = 1.34's, 81.047'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.315'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.291'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 302.822'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32056.385'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.047'MB/sec zip pipeline throughput = 81.047'MB/sec entries = 927932 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 36.22 usrkeys = 927932 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.56 avg-zval = 36.22 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 784041 bytes = 2646160 } seq expand size = 5488287 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5110 value = 3.2199 dict = 1.00 all = 3.4186 } Zip/UnZip{ index = 0.0951 value = 0.3106 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 0.789481 GB avg = 0.119 KB (by entry num) total key len = 0.185756 GB avg = 0.028 KB total ukey len = 0.132683 GB avg = 0.020 KB total ukey num = 0.006634154 Billion total entry num = 0.006634154 Billion write speed all = 0.034310103 MB/sec (with seq num) write speed all = 0.032442919 MB/sec (without seq num) 2024/11/22-16:46:07.345404 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215477: 927932 keys, 39261148 bytes, temperature: kWarm 2024/11/22-16:46:07.345470 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265167345437, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215477, "file_size": 39261148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3391789830, "table_properties": {"data_size": 33614336, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982096, "raw_average_key_size": 28, "raw_value_size": 108235712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265164, "oldest_key_time": 0, "file_creation_time": 1732265164, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:07.381908 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst) /dev/shm/ToplingTemp/Topling-205010-EPBQV4 2024/11/22-16:46:08.328771 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst): first pass time = 0.95's, 141.757'MB/sec 2024/11/22-16:46:09.738352 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst): old prealloc_size = 147639500, real_size = 36112832 2024/11/22-16:46:09.738893 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923873 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0341 GB avg-val = 117.28 avg-zval = 36.94 2024/11/22-16:46:09.739283 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:10.493897 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215478.sst): second pass time = 1.41's, 76.903'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 50.797'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3390.829'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.367'MB/sec write SST data time = 0.13's, 306.296'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32534.515'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.903'MB/sec zip pipeline throughput = 76.903'MB/sec entries = 923873 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 36.94 usrkeys = 923873 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.11 avg-zval = 36.94 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 373465 bytes = 1260464 } seq expand size = 2614255 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5135 value = 3.1752 dict = 1.00 all = 3.4972 } Zip/UnZip{ index = 0.0951 value = 0.3149 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 0.900445 GB avg = 0.119 KB (by entry num) total key len = 0.211625 GB avg = 0.028 KB total ukey len = 0.151161 GB avg = 0.020 KB total ukey num = 0.007558027 Billion total entry num = 0.007558027 Billion write speed all = 0.039119648 MB/sec (with seq num) write speed all = 0.036992678 MB/sec (without seq num) 2024/11/22-16:46:10.530747 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215478: 923873 keys, 38378921 bytes, temperature: kWarm 2024/11/22-16:46:10.530819 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265170530786, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215478, "file_size": 38378921, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3344763295, "table_properties": {"data_size": 34124144, "index_size": 1757504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868444, "raw_average_key_size": 28, "raw_value_size": 108349414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923873, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265167, "oldest_key_time": 0, "file_creation_time": 1732265167, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:10.566085 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst) /dev/shm/ToplingTemp/Topling-205010-wYVCi5 2024/11/22-16:46:11.383818 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst): first pass time = 0.82's, 164.143'MB/sec 2024/11/22-16:46:12.724065 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst): old prealloc_size = 147639500, real_size = 33755280 2024/11/22-16:46:12.724596 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921468 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0318 GB avg-val = 117.66 avg-zval = 34.47 2024/11/22-16:46:12.725358 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:13.647985 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215479.sst): second pass time = 1.34's, 80.933'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3461.890'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 298.702'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33689.797'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.933'MB/sec zip pipeline throughput = 80.933'MB/sec entries = 921468 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.66 avg-zval = 34.47 usrkeys = 921468 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.66 avg-zval = 34.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921468 bytes = 3109968 } seq expand size = 6450276 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4454 value = 3.4136 dict = 1.00 all = 3.5578 } Zip/UnZip{ index = 0.0957 value = 0.2929 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 1.015312 GB avg = 0.120 KB (by entry num) total key len = 0.237426 GB avg = 0.028 KB total ukey len = 0.169590 GB avg = 0.020 KB total ukey num = 0.008479495 Billion total entry num = 0.008479495 Billion write speed all = 0.044063087 MB/sec (with seq num) write speed all = 0.041677064 MB/sec (without seq num) 2024/11/22-16:46:13.684725 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215479: 921468 keys, 37724450 bytes, temperature: kWarm 2024/11/22-16:46:13.684799 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265173684757, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215479, "file_size": 37724450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701510, "largest_seqno": 3391789559, "table_properties": {"data_size": 31760384, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25801104, "raw_average_key_size": 28, "raw_value_size": 108416700, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921468, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265170, "oldest_key_time": 0, "file_creation_time": 1732265170, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:13.723279 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst) /dev/shm/ToplingTemp/Topling-205010-cnmcc8 2024/11/22-16:46:14.519858 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst): first pass time = 0.80's, 168.503'MB/sec 2024/11/22-16:46:15.836317 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst): old prealloc_size = 147639500, real_size = 33622976 2024/11/22-16:46:15.836867 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923573 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.32 avg-zval = 34.25 2024/11/22-16:46:15.837640 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-16:46:16.738100 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215480.sst): second pass time = 1.32's, 82.354'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.693'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3374.587'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 293.123'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34618.233'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.354'MB/sec zip pipeline throughput = 82.354'MB/sec entries = 923573 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 34.25 usrkeys = 923573 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923573 bytes = 3117072 } seq expand size = 6465011 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5131 value = 3.4253 dict = 1.00 all = 3.5698 } Zip/UnZip{ index = 0.0951 value = 0.2919 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 1.130135 GB avg = 0.120 KB (by entry num) total key len = 0.263286 GB avg = 0.028 KB total ukey len = 0.188061 GB avg = 0.020 KB total ukey num = 0.009403068 Billion total entry num = 0.009403068 Billion write speed all = 0.049006055 MB/sec (with seq num) write speed all = 0.046360437 MB/sec (without seq num) 2024/11/22-16:46:16.774175 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215480: 923573 keys, 37597640 bytes, temperature: kWarm 2024/11/22-16:46:16.774238 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265176774205, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215480, "file_size": 37597640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701530, "largest_seqno": 3391789586, "table_properties": {"data_size": 31634928, "index_size": 1756992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860044, "raw_average_key_size": 28, "raw_value_size": 108357690, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265173, "oldest_key_time": 0, "file_creation_time": 1732265173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:16.811606 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst) /dev/shm/ToplingTemp/Topling-205010-Wbcx64 2024/11/22-16:46:17.165487 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst): first pass time = 0.35's, 250.713'MB/sec 2024/11/22-16:46:18.042515 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst): old prealloc_size = 147639500, real_size = 22355520 2024/11/22-16:46:18.042900 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 609726 prefix = 4 raw-key = 0.0098 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0716 GB zip-val = 0.0210 GB avg-val = 117.50 avg-zval = 34.51 2024/11/22-16:46:18.043406 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.089 GB 2024/11/22-16:46:18.678346 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00/215481.sst): second pass time = 0.88's, 81.748'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 39.295'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.147'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.277'MB/sec write SST data time = 0.09's, 293.607'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.719'MB, throughput = 21279.865'MB/sec zip my value time = 0.88's, unzip length = 0.072'GB zip my value throughput = 81.748'MB/sec zip pipeline throughput = 81.748'MB/sec entries = 609726 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 34.51 usrkeys = 609726 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.50 avg-zval = 34.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 609726 bytes = 2057840 } seq expand size = 4268082 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0122 GB value = 0.0716 GB dict = 0.72 MB all = 0.0887 GB } __ZipSize{ index = 0.0012 GB value = 0.0210 GB dict = 0.72 MB all = 0.0250 GB } UnZip/Zip{ index = 10.5108 value = 3.4045 dict = 1.00 all = 3.5507 } Zip/UnZip{ index = 0.0951 value = 0.2937 dict = 1.00 all = 0.2816 } ---------------------------- total value len = 1.206044 GB avg = 0.120 KB (by entry num) total key len = 0.280358 GB avg = 0.028 KB total ukey len = 0.200256 GB avg = 0.020 KB total ukey num = 0.010012794 Billion total entry num = 0.010012794 Billion write speed all = 0.052272601 MB/sec (with seq num) write speed all = 0.049455625 MB/sec (without seq num) 2024/11/22-16:46:18.703736 140024844146240 [compaction_job.cc:1948] [default] [JOB 4994] Generated table #215481: 609726 keys, 24984469 bytes, temperature: kWarm 2024/11/22-16:46:18.703798 140024844146240 EVENT_LOG_v1 {"time_micros": 1732265178703762, "cf_name": "default", "job": 4994, "event": "table_file_creation", "file_number": 215481, "file_size": 24984469, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701561, "largest_seqno": 3391789606, "table_properties": {"data_size": 21042736, "index_size": 1160192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17072328, "raw_average_key_size": 28, "raw_value_size": 71640820, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 609726, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732265176, "oldest_key_time": 0, "file_creation_time": 1732265176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 215481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-16:46:18.726432 140024844146240 [compaction_job.cc:703] job-04994: subcompact[0], size: 0.406776 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-04994/att-00 [215471,215472,215473,215474,215475,215476,215477,215478,215479,215480,215481] 2024/11/22-16:46:18.797217 140024844146240 (Original Log Time 2024/11/22-16:45:45.261420) EVENT_LOG_v1 {"time_micros": 1732265145261388, "job": 4994, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [215422], "files_L3": [215467, 215468, 215372, 215373, 215374, 215375, 215376, 215377, 215378, 215339, 215340, 215433], "score": 0.978264, "input_data_size": 430038806, "oldest_snapshot_seqno": 3392491005} 2024/11/22-16:46:18.797220 140024844146240 (Original Log Time 2024/11/22-16:46:18.796600) [compaction_job.cc:2022] [default] [JOB 4994] Compacted 1@2 + 12@3 files to L3 => 406775947 bytes 2024/11/22-16:46:18.797221 140024844146240 (Original Log Time 2024/11/22-16:46:18.797155) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 12.9 rd, 12.2 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(27.1, 383.0 +0.0 blob) out(387.9 +0.0 blob), read-write-amplify(29.5) write-amplify(14.3) OK, records in: 10106565, records dropped: 93771 output_compression: Snappy 2024/11/22-16:46:18.797228 140024844146240 (Original Log Time 2024/11/22-16:46:18.797199) EVENT_LOG_v1 {"time_micros": 1732265178797179, "cf": "default", "job": 4994, "event": "compaction_finished", "compaction_time_micros": 33465017, "compaction_time_cpu_micros": 14832854, "output_level": 3, "num_output_files": 11, "total_output_size": 406775947, "num_input_records": 10106565, "num_output_records": 10012794, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 252868653, "file_range_sync_nanos": 0, "file_fsync_nanos": 11035, "file_prepare_write_nanos": 25040, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-16:46:18.914197 140024844146240 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732265145,"executesMs":33668,"compactionJobId":4994,"attempt":0,"compactionInputRawBytes":1444206691,"compactionInputZipBytes":430038806,"compactionOutputRawBytes":0,"compactionOutputZipBytes":406775947,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-69b3e2aa64d929a2a7244b5cc268b5b4-774307649df35c3e-00"}}, response = 2024/11/22-16:46:18.914266 140024844146240 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-04994/att-00: olev 3, work 33.668 s, result 4.910 ms, install 0.645 ms, input{raw 1.345 GiB zip 410.117 MiB}