2024/11/22-18:56:10.747062 140024928073280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00: fork to child time = 0.009540 sec 2024/11/22-18:56:10.747943 140024928073280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 2024/11/22-18:56:10.755631 140024928073280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 2024/11/22-18:56:10.756642 140024928073280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-18:56:10.756928 140024928073280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-18:56:10.811586 140024928073280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08157/att-00/MANIFEST-206347 2024/11/22-18:56:10.846801 140024928073280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08157/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 221813, last_sequence is 3483031725, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 221608 2024/11/22-18:56:10.846810 140024928073280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-18:56:10.846922 140024928073280 [version_set.cc:5721] Creating manifest 221813 2024/11/22-18:56:10.879392 140024928073280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-18:56:10.902057 140024928073280 [compaction_job.cc:2437] [default] [JOB 8157] Compacting 1@2 + 15@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/22-18:56:10.902076 140024928073280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6192 Base level 2, inputs: [221758(27MB)], [221810(35MB) 221811(4821KB) 221699(35MB) 221708(35MB) 221709(35MB) 221710(35MB) 221711(35MB) 221712(34MB) 221713(36MB) 221714(35MB) 221715(35MB) 221716(36MB) 221717(35MB) 221718(33MB) 221785(36MB)] 2024/11/22-18:56:10.909438 140024928073280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:56:10.909472 140024928073280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:56:10.909493 140024928073280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-18:56:10.909499 140024928073280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:56:10.909510 140024928073280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-18:56:11.351612 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst) /dev/shm/ToplingTemp/Topling-249248-J9GBbc 2024/11/22-18:56:11.886373 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst): first pass time = 0.53's, 251.009'MB/sec 2024/11/22-18:56:13.206660 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst): old prealloc_size = 147639500, real_size = 32837376 2024/11/22-18:56:13.207237 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0308 GB avg-val = 117.46 avg-zval = 33.41 2024/11/22-18:56:13.208064 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:14.092153 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221814.sst): second pass time = 1.32's, 82.138'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.701'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.876'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 287.007'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33980.363'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.138'MB/sec zip pipeline throughput = 82.138'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.46 avg-zval = 33.41 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.46 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3229456 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3750 value = 3.5157 dict = 1.00 all = 3.6349 } Zip/UnZip{ index = 0.0964 value = 0.2844 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 0.114841 GB avg = 0.124 KB (by entry num) total key len = 0.025836 GB avg = 0.028 KB total ukey len = 0.018454 GB avg = 0.020 KB total ukey num = 0.000922697 Billion total entry num = 0.000922697 Billion write speed all = 0.003882771 MB/sec (with seq num) write speed all = 0.003679034 MB/sec (without seq num) 2024/11/22-18:56:14.124817 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221814: 922697 keys, 36925226 bytes, temperature: kWarm 2024/11/22-18:56:14.124912 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272974124855, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221814, "file_size": 36925226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3482512504, "table_properties": {"data_size": 30827824, "index_size": 1778688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382258, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272971, "oldest_key_time": 0, "file_creation_time": 1732272971, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221814, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:14.167378 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst) /dev/shm/ToplingTemp/Topling-249248-yacqp8 2024/11/22-18:56:15.019801 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst): first pass time = 0.85's, 157.462'MB/sec 2024/11/22-18:56:16.353284 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst): old prealloc_size = 147639500, real_size = 33447952 2024/11/22-18:56:16.353872 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922953 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0315 GB avg-val = 117.42 avg-zval = 34.09 2024/11/22-18:56:16.354767 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:17.269057 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221815.sst): second pass time = 1.33's, 81.348'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.864'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3057.595'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 294.877'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29951.149'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.348'MB/sec zip pipeline throughput = 81.348'MB/sec entries = 922953 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 34.09 usrkeys = 922953 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922953 bytes = 3114976 } seq expand size = 6460671 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5118 value = 3.4447 dict = 1.00 all = 3.5867 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.229677 GB avg = 0.124 KB (by entry num) total key len = 0.051678 GB avg = 0.028 KB total ukey len = 0.036913 GB avg = 0.020 KB total ukey num = 0.001845650 Billion total entry num = 0.001845650 Billion write speed all = 0.007764909 MB/sec (with seq num) write speed all = 0.007357415 MB/sec (without seq num) 2024/11/22-18:56:17.301313 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221815: 922953 keys, 37420764 bytes, temperature: kWarm 2024/11/22-18:56:17.301375 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272977301342, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221815, "file_size": 37420764, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3482512515, "table_properties": {"data_size": 31460992, "index_size": 1756032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842684, "raw_average_key_size": 28, "raw_value_size": 108375054, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922953, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272974, "oldest_key_time": 0, "file_creation_time": 1732272974, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221815, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:17.344153 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst) /dev/shm/ToplingTemp/Topling-249248-NWEPmb 2024/11/22-18:56:18.186960 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst): first pass time = 0.84's, 159.258'MB/sec 2024/11/22-18:56:19.481823 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst): old prealloc_size = 147639500, real_size = 33431792 2024/11/22-18:56:19.482372 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924492 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.18 avg-zval = 34.01 2024/11/22-18:56:19.483179 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:20.416611 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221816.sst): second pass time = 1.29's, 83.707'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.019'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3321.823'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 291.701'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30218.175'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.707'MB/sec zip pipeline throughput = 83.707'MB/sec entries = 924492 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 34.01 usrkeys = 924492 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924492 bytes = 3120176 } seq expand size = 6471444 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5136 value = 3.4455 dict = 1.00 all = 3.5879 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 0.344480 GB avg = 0.124 KB (by entry num) total key len = 0.077564 GB avg = 0.028 KB total ukey len = 0.055403 GB avg = 0.020 KB total ukey num = 0.002770142 Billion total entry num = 0.002770142 Billion write speed all = 0.011646677 MB/sec (with seq num) write speed all = 0.011035121 MB/sec (without seq num) 2024/11/22-18:56:20.447893 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221816: 924492 keys, 37408392 bytes, temperature: kWarm 2024/11/22-18:56:20.447960 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272980447926, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221816, "file_size": 37408392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3482513312, "table_properties": {"data_size": 31441824, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885776, "raw_average_key_size": 28, "raw_value_size": 108332080, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272977, "oldest_key_time": 0, "file_creation_time": 1732272977, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221816, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:20.485572 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst) /dev/shm/ToplingTemp/Topling-249248-ZtSlv8 2024/11/22-18:56:21.300070 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst): first pass time = 0.81's, 164.794'MB/sec 2024/11/22-18:56:22.616126 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst): old prealloc_size = 147639500, real_size = 33587024 2024/11/22-18:56:22.616662 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924874 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.12 avg-zval = 34.16 2024/11/22-18:56:22.619270 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:23.521425 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221817.sst): second pass time = 1.32's, 82.353'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.494'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3432.179'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.298'MB/sec write SST data time = 0.13's, 289.288'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33081.245'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.353'MB/sec zip pipeline throughput = 82.353'MB/sec entries = 924874 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 34.16 usrkeys = 924874 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 34.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924874 bytes = 3121472 } seq expand size = 6474118 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5134 value = 3.4283 dict = 1.00 all = 3.5730 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 0.459276 GB avg = 0.124 KB (by entry num) total key len = 0.103460 GB avg = 0.028 KB total ukey len = 0.073900 GB avg = 0.020 KB total ukey num = 0.003695016 Billion total entry num = 0.003695016 Billion write speed all = 0.015527862 MB/sec (with seq num) write speed all = 0.014712195 MB/sec (without seq num) 2024/11/22-18:56:23.553819 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221817: 924874 keys, 37564214 bytes, temperature: kWarm 2024/11/22-18:56:23.553884 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272983553852, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221817, "file_size": 37564214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3482512548, "table_properties": {"data_size": 31596224, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896472, "raw_average_key_size": 28, "raw_value_size": 108321364, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924874, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272980, "oldest_key_time": 0, "file_creation_time": 1732272980, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221817, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:23.594245 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst) /dev/shm/ToplingTemp/Topling-249248-0jGMu8 2024/11/22-18:56:24.512144 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst): first pass time = 0.92's, 146.230'MB/sec 2024/11/22-18:56:25.832742 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst): old prealloc_size = 147639500, real_size = 33674752 2024/11/22-18:56:25.833298 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922956 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.42 avg-zval = 34.33 2024/11/22-18:56:25.834055 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:26.770260 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221818.sst): second pass time = 1.32's, 82.106'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.157'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.441'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 293.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31050.087'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.106'MB/sec zip pipeline throughput = 82.106'MB/sec entries = 922956 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 34.33 usrkeys = 922956 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 34.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922956 bytes = 3114992 } seq expand size = 6460692 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5138 value = 3.4201 dict = 1.00 all = 3.5651 } Zip/UnZip{ index = 0.0951 value = 0.2924 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.574112 GB avg = 0.124 KB (by entry num) total key len = 0.129303 GB avg = 0.028 KB total ukey len = 0.092359 GB avg = 0.020 KB total ukey num = 0.004617972 Billion total entry num = 0.004617972 Billion write speed all = 0.019407932 MB/sec (with seq num) write speed all = 0.018388615 MB/sec (without seq num) 2024/11/22-18:56:26.801451 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221818: 922956 keys, 37647812 bytes, temperature: kWarm 2024/11/22-18:56:26.801518 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272986801486, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221818, "file_size": 37647812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3482512568, "table_properties": {"data_size": 31688112, "index_size": 1755712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842768, "raw_average_key_size": 28, "raw_value_size": 108374988, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922956, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272983, "oldest_key_time": 0, "file_creation_time": 1732272983, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221818, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:26.840130 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst) /dev/shm/ToplingTemp/Topling-249248-PbCTsa 2024/11/22-18:56:27.627390 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst): first pass time = 0.79's, 170.497'MB/sec 2024/11/22-18:56:28.966520 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst): old prealloc_size = 147639500, real_size = 33450288 2024/11/22-18:56:28.967097 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927403 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.72 avg-zval = 33.92 2024/11/22-18:56:28.967821 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:29.877763 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221819.sst): second pass time = 1.34's, 80.875'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.892'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.153'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 283.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29827.989'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.875'MB/sec zip pipeline throughput = 80.875'MB/sec entries = 927403 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.72 avg-zval = 33.92 usrkeys = 927403 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.01 avg-zval = 33.92 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 832239 bytes = 2808816 } seq expand size = 5825673 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5142 value = 3.4415 dict = 1.00 all = 3.6020 } Zip/UnZip{ index = 0.0951 value = 0.2906 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.688188 GB avg = 0.124 KB (by entry num) total key len = 0.155271 GB avg = 0.028 KB total ukey len = 0.110908 GB avg = 0.020 KB total ukey num = 0.005545375 Billion total entry num = 0.005545375 Billion write speed all = 0.023269880 MB/sec (with seq num) write speed all = 0.022045965 MB/sec (without seq num) 2024/11/22-18:56:29.909800 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221819: 927403 keys, 37261630 bytes, temperature: kWarm 2024/11/22-18:56:29.909876 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272989909833, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221819, "file_size": 37261630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3482513197, "table_properties": {"data_size": 31454176, "index_size": 1764096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967284, "raw_average_key_size": 28, "raw_value_size": 108250542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927403, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272986, "oldest_key_time": 0, "file_creation_time": 1732272986, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221819, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:29.946515 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst) /dev/shm/ToplingTemp/Topling-249248-jOVKp9 2024/11/22-18:56:30.763639 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst): first pass time = 0.82's, 164.265'MB/sec 2024/11/22-18:56:32.131788 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst): old prealloc_size = 147639500, real_size = 34988208 2024/11/22-18:56:32.132405 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081665 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.08 avg-zval = 30.19 2024/11/22-18:56:32.132450 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:32.758043 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221820.sst): second pass time = 1.37's, 76.008'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3527.251'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.556'MB/sec write SST data time = 0.14's, 257.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 24784.927'MB/sec zip my value time = 1.37's, unzip length = 0.104'GB zip my value throughput = 76.008'MB/sec zip pipeline throughput = 76.008'MB/sec entries = 1081665 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.19 usrkeys = 1081665 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.19 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5155 value = 3.1822 dict = 1.00 all = 3.7529 } Zip/UnZip{ index = 0.0951 value = 0.3142 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 0.792119 GB avg = 0.120 KB (by entry num) total key len = 0.185557 GB avg = 0.028 KB total ukey len = 0.132541 GB avg = 0.020 KB total ukey num = 0.006627040 Billion total entry num = 0.006627040 Billion write speed all = 0.026970624 MB/sec (with seq num) write speed all = 0.025508092 MB/sec (without seq num) 2024/11/22-18:56:32.789353 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221820: 1081665 keys, 35763291 bytes, temperature: kWarm 2024/11/22-18:56:32.789415 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272992789383, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221820, "file_size": 35763291, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32660320, "index_size": 2057280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30286620, "raw_average_key_size": 28, "raw_value_size": 103931135, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081665, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272989, "oldest_key_time": 0, "file_creation_time": 1732272989, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221820, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:32.826214 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst) /dev/shm/ToplingTemp/Topling-249248-3NYCA9 2024/11/22-18:56:33.645947 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst): first pass time = 0.82's, 163.741'MB/sec 2024/11/22-18:56:34.974707 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst): old prealloc_size = 147639500, real_size = 33980736 2024/11/22-18:56:34.975261 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924579 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.17 avg-zval = 34.60 2024/11/22-18:56:34.975934 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:35.845533 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221821.sst): second pass time = 1.33's, 81.567'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.041'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.674'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 287.268'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29524.860'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.567'MB/sec zip pipeline throughput = 81.567'MB/sec entries = 924579 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 34.60 usrkeys = 924579 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.56 avg-zval = 34.60 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 713005 bytes = 2495536 } seq expand size = 4991035 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5127 value = 3.3863 dict = 1.00 all = 3.5810 } Zip/UnZip{ index = 0.0951 value = 0.2953 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.905440 GB avg = 0.120 KB (by entry num) total key len = 0.211445 GB avg = 0.028 KB total ukey len = 0.151032 GB avg = 0.020 KB total ukey num = 0.007551619 Billion total entry num = 0.007551619 Billion write speed all = 0.030808280 MB/sec (with seq num) write speed all = 0.029141842 MB/sec (without seq num) 2024/11/22-18:56:35.878199 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221821: 924579 keys, 37480532 bytes, temperature: kWarm 2024/11/22-18:56:35.878262 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272995878229, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221821, "file_size": 37480532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3482512248, "table_properties": {"data_size": 31990448, "index_size": 1758976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25888212, "raw_average_key_size": 28, "raw_value_size": 108329602, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924579, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272992, "oldest_key_time": 0, "file_creation_time": 1732272992, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221821, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:35.921466 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst) /dev/shm/ToplingTemp/Topling-249248-fX2iSb 2024/11/22-18:56:36.783059 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst): first pass time = 0.86's, 155.787'MB/sec 2024/11/22-18:56:38.106155 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst): old prealloc_size = 147639500, real_size = 33656432 2024/11/22-18:56:38.106675 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922989 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.42 avg-zval = 34.31 2024/11/22-18:56:38.107895 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:39.038520 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221822.sst): second pass time = 1.32's, 81.956'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.360'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3524.399'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.287'MB/sec write SST data time = 0.13's, 292.372'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30386.644'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.956'MB/sec zip pipeline throughput = 81.956'MB/sec entries = 922989 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.42 avg-zval = 34.31 usrkeys = 922989 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.42 avg-zval = 34.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922989 bytes = 3115104 } seq expand size = 6460923 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5122 value = 3.4220 dict = 1.00 all = 3.5668 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 1.020275 GB avg = 0.120 KB (by entry num) total key len = 0.237289 GB avg = 0.028 KB total ukey len = 0.169492 GB avg = 0.020 KB total ukey num = 0.008474608 Billion total entry num = 0.008474608 Billion write speed all = 0.034685722 MB/sec (with seq num) write speed all = 0.032815770 MB/sec (without seq num) 2024/11/22-18:56:39.072096 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221822: 922989 keys, 37629620 bytes, temperature: kWarm 2024/11/22-18:56:39.072165 140024928073280 EVENT_LOG_v1 {"time_micros": 1732272999072131, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221822, "file_size": 37629620, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3482512109, "table_properties": {"data_size": 31669472, "index_size": 1756032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843692, "raw_average_key_size": 28, "raw_value_size": 108374066, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922989, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272995, "oldest_key_time": 0, "file_creation_time": 1732272995, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221822, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:39.106159 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst) /dev/shm/ToplingTemp/Topling-249248-QFl3V8 2024/11/22-18:56:39.989254 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst): first pass time = 0.88's, 151.993'MB/sec 2024/11/22-18:56:41.337000 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst): old prealloc_size = 147639500, real_size = 33850640 2024/11/22-18:56:41.337511 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922882 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0319 GB avg-val = 117.43 avg-zval = 34.53 2024/11/22-18:56:41.338314 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:42.268022 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221823.sst): second pass time = 1.35's, 80.455'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.751'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3574.404'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 302.996'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32523.228'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.455'MB/sec zip pipeline throughput = 80.455'MB/sec entries = 922882 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.43 avg-zval = 34.53 usrkeys = 922882 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.43 avg-zval = 34.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922882 bytes = 3230096 } seq expand size = 6460174 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5121 value = 3.4013 dict = 1.00 all = 3.5377 } Zip/UnZip{ index = 0.0951 value = 0.2940 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 1.135112 GB avg = 0.121 KB (by entry num) total key len = 0.263130 GB avg = 0.028 KB total ukey len = 0.187950 GB avg = 0.020 KB total ukey num = 0.009397490 Billion total entry num = 0.009397490 Billion write speed all = 0.038562423 MB/sec (with seq num) write speed all = 0.036489019 MB/sec (without seq num) 2024/11/22-18:56:42.300809 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221823: 922882 keys, 37938880 bytes, temperature: kWarm 2024/11/22-18:56:42.300875 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273002300841, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221823, "file_size": 37938880, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3482510981, "table_properties": {"data_size": 31863872, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25840696, "raw_average_key_size": 28, "raw_value_size": 108377112, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922882, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732272999, "oldest_key_time": 0, "file_creation_time": 1732272999, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:42.339240 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst) /dev/shm/ToplingTemp/Topling-249248-GkNLO8 2024/11/22-18:56:43.133181 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst): first pass time = 0.79's, 169.062'MB/sec 2024/11/22-18:56:44.461847 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst): old prealloc_size = 147639500, real_size = 33440848 2024/11/22-18:56:44.462377 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923044 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.41 avg-zval = 34.05 2024/11/22-18:56:44.463191 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:45.371582 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221824.sst): second pass time = 1.33's, 81.637'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3492.887'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 295.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28189.388'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.637'MB/sec zip pipeline throughput = 81.637'MB/sec entries = 923044 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.41 avg-zval = 34.05 usrkeys = 923044 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.41 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923044 bytes = 3230672 } seq expand size = 6461308 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3759 value = 3.4480 dict = 1.00 all = 3.5763 } Zip/UnZip{ index = 0.0964 value = 0.2900 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 1.249946 GB avg = 0.121 KB (by entry num) total key len = 0.288975 GB avg = 0.028 KB total ukey len = 0.206411 GB avg = 0.020 KB total ukey num = 0.010320534 Billion total entry num = 0.010320534 Billion write speed all = 0.042438610 MB/sec (with seq num) write speed all = 0.040161746 MB/sec (without seq num) 2024/11/22-18:56:45.407661 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221824: 923044 keys, 37529306 bytes, temperature: kWarm 2024/11/22-18:56:45.407727 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273005407693, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221824, "file_size": 37529306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3482511000, "table_properties": {"data_size": 31430720, "index_size": 1779200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25845232, "raw_average_key_size": 28, "raw_value_size": 108372516, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923044, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732273002, "oldest_key_time": 0, "file_creation_time": 1732273002, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:45.445465 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst) /dev/shm/ToplingTemp/Topling-249248-dZd2q9 2024/11/22-18:56:46.239304 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst): first pass time = 0.79's, 169.084'MB/sec 2024/11/22-18:56:47.553831 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst): old prealloc_size = 147639500, real_size = 33647808 2024/11/22-18:56:47.554365 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923001 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.41 avg-zval = 34.30 2024/11/22-18:56:47.555139 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:48.489374 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221825.sst): second pass time = 1.31's, 82.485'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.214'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3445.582'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.474'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32198.603'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.485'MB/sec zip pipeline throughput = 82.485'MB/sec entries = 923001 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 34.30 usrkeys = 923001 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 34.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923001 bytes = 3115136 } seq expand size = 6461007 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5120 value = 3.4230 dict = 1.00 all = 3.5676 } Zip/UnZip{ index = 0.0951 value = 0.2921 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 1.364780 GB avg = 0.121 KB (by entry num) total key len = 0.314819 GB avg = 0.028 KB total ukey len = 0.224871 GB avg = 0.020 KB total ukey num = 0.011243535 Billion total entry num = 0.011243535 Billion write speed all = 0.046314111 MB/sec (with seq num) write speed all = 0.043833832 MB/sec (without seq num) 2024/11/22-18:56:48.521532 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221825: 923001 keys, 37621064 bytes, temperature: kWarm 2024/11/22-18:56:48.521608 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273008521564, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221825, "file_size": 37621064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3482513171, "table_properties": {"data_size": 31660784, "index_size": 1756096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844028, "raw_average_key_size": 28, "raw_value_size": 108373826, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923001, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732273005, "oldest_key_time": 0, "file_creation_time": 1732273005, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:48.557916 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst) /dev/shm/ToplingTemp/Topling-249248-WHkT19 2024/11/22-18:56:49.348919 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst): first pass time = 0.79's, 169.689'MB/sec 2024/11/22-18:56:50.645634 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst): old prealloc_size = 147639500, real_size = 33590800 2024/11/22-18:56:50.646184 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922833 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.44 avg-zval = 34.25 2024/11/22-18:56:50.646919 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:51.587896 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221826.sst): second pass time = 1.30's, 83.643'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.593'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 285.672'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31043.112'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.643'MB/sec zip pipeline throughput = 83.643'MB/sec entries = 922833 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 34.25 usrkeys = 922833 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 34.25 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 822999 bytes = 2880512 } seq expand size = 5760993 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5116 value = 3.4293 dict = 1.00 all = 3.5817 } Zip/UnZip{ index = 0.0951 value = 0.2916 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.478920 GB avg = 0.122 KB (by entry num) total key len = 0.340658 GB avg = 0.028 KB total ukey len = 0.243327 GB avg = 0.020 KB total ukey num = 0.012166368 Billion total entry num = 0.012166368 Billion write speed all = 0.050169668 MB/sec (with seq num) write speed all = 0.047486046 MB/sec (without seq num) 2024/11/22-18:56:51.622478 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221826: 922833 keys, 37473706 bytes, temperature: kWarm 2024/11/22-18:56:51.622552 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273011622509, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221826, "file_size": 37473706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3482511029, "table_properties": {"data_size": 31604096, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25839324, "raw_average_key_size": 28, "raw_value_size": 108378434, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922833, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732273008, "oldest_key_time": 0, "file_creation_time": 1732273008, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:51.658154 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst) /dev/shm/ToplingTemp/Topling-249248-AxIS48 2024/11/22-18:56:52.569530 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst): first pass time = 0.91's, 147.278'MB/sec 2024/11/22-18:56:53.900192 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst): old prealloc_size = 147639500, real_size = 33468352 2024/11/22-18:56:53.900738 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924983 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.10 avg-zval = 34.03 2024/11/22-18:56:53.901413 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-18:56:54.840389 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221827.sst): second pass time = 1.33's, 81.486'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.457'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.959'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 283.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32578.876'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.486'MB/sec zip pipeline throughput = 81.486'MB/sec entries = 924983 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 34.03 usrkeys = 924983 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 34.03 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 750916 bytes = 2628224 } seq expand size = 5256412 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5138 value = 3.4411 dict = 1.00 all = 3.6180 } Zip/UnZip{ index = 0.0951 value = 0.2906 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.592495 GB avg = 0.122 KB (by entry num) total key len = 0.366558 GB avg = 0.028 KB total ukey len = 0.261827 GB avg = 0.020 KB total ukey num = 0.013091351 Billion total entry num = 0.013091351 Billion write speed all = 0.054010426 MB/sec (with seq num) write speed all = 0.051123032 MB/sec (without seq num) 2024/11/22-18:56:54.873378 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221827: 924983 keys, 37096954 bytes, temperature: kWarm 2024/11/22-18:56:54.873443 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273014873411, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221827, "file_size": 37096954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3482511063, "table_properties": {"data_size": 31477360, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899524, "raw_average_key_size": 28, "raw_value_size": 108318234, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924983, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732273011, "oldest_key_time": 0, "file_creation_time": 1732273011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:54.909985 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst) /dev/shm/ToplingTemp/Topling-249248-SHGJ5a 2024/11/22-18:56:55.049735 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst): first pass time = 0.14's, 235.808'MB/sec 2024/11/22-18:56:55.362857 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst): old prealloc_size = 147639500, real_size = 9479248 2024/11/22-18:56:55.363039 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 227279 prefix = 4 raw-key = 0.0036 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0266 GB zip-val = 0.0090 GB avg-val = 116.95 avg-zval = 39.54 2024/11/22-18:56:55.363249 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.033 GB 2024/11/22-18:56:55.574929 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00/221828.sst): second pass time = 0.31's, 85.047'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 49.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2764.089'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.317'MB/sec write SST data time = 0.03's, 321.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.274'MB, throughput = 9418.854'MB/sec zip my value time = 0.31's, unzip length = 0.027'GB zip my value throughput = 85.047'MB/sec zip pipeline throughput = 85.047'MB/sec entries = 227279 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 39.54 usrkeys = 227279 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.95 avg-zval = 39.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 227279 bytes = 767088 } seq expand size = 1590953 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0045 GB value = 0.0266 GB dict = 0.27 MB all = 0.0329 GB } __ZipSize{ index = 0.0004 GB value = 0.0090 GB dict = 0.27 MB all = 0.0105 GB } UnZip/Zip{ index = 10.4142 value = 2.9581 dict = 1.00 all = 3.1478 } Zip/UnZip{ index = 0.0960 value = 0.3381 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 1.620667 GB avg = 0.122 KB (by entry num) total key len = 0.372922 GB avg = 0.028 KB total ukey len = 0.266373 GB avg = 0.020 KB total ukey num = 0.013318630 Billion total entry num = 0.013318630 Billion write speed all = 0.054961458 MB/sec (with seq num) write speed all = 0.052023996 MB/sec (without seq num) 2024/11/22-18:56:55.588912 140024928073280 [compaction_job.cc:1948] [default] [JOB 8157] Generated table #221828: 227279 keys, 10465987 bytes, temperature: kWarm 2024/11/22-18:56:55.588973 140024928073280 EVENT_LOG_v1 {"time_micros": 1732273015588941, "cf_name": "default", "job": 8157, "event": "table_file_creation", "file_number": 221828, "file_size": 10465987, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3482512857, "table_properties": {"data_size": 8985792, "index_size": 436480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6363812, "raw_average_key_size": 28, "raw_value_size": 26581206, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 227279, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732273014, "oldest_key_time": 0, "file_creation_time": 1732273014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 221828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-18:56:55.602636 140024928073280 [compaction_job.cc:703] job-08157: subcompact[0], size: 0.533227 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08157/att-00 [221814,221815,221816,221817,221818,221819,221820,221821,221822,221823,221824,221825,221826,221827,221828] 2024/11/22-18:56:55.693317 140024928073280 (Original Log Time 2024/11/22-18:56:10.902146) EVENT_LOG_v1 {"time_micros": 1732272970902092, "job": 8157, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [221758], "files_L3": [221810, 221811, 221699, 221708, 221709, 221710, 221711, 221712, 221713, 221714, 221715, 221716, 221717, 221718, 221785], "score": 0.980383, "input_data_size": 553802042, "oldest_snapshot_seqno": 3483031670} 2024/11/22-18:56:55.693320 140024928073280 (Original Log Time 2024/11/22-18:56:55.691524) [compaction_job.cc:2022] [default] [JOB 8157] Compacted 1@2 + 15@3 files to L3 => 533227378 bytes 2024/11/22-18:56:55.693322 140024928073280 (Original Log Time 2024/11/22-18:56:55.693250) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.4 rd, 11.9 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 501.0 +0.0 blob) out(508.5 +0.0 blob), read-write-amplify(38.2) write-amplify(18.8) OK, records in: 13428037, records dropped: 109407 output_compression: Snappy 2024/11/22-18:56:55.693330 140024928073280 (Original Log Time 2024/11/22-18:56:55.693296) EVENT_LOG_v1 {"time_micros": 1732273015693273, "cf": "default", "job": 8157, "event": "compaction_finished", "compaction_time_micros": 44700498, "compaction_time_cpu_micros": 20677834, "output_level": 3, "num_output_files": 15, "total_output_size": 533227378, "num_input_records": 13428037, "num_output_records": 13318630, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 338951440, "file_range_sync_nanos": 0, "file_fsync_nanos": 14865, "file_prepare_write_nanos": 33552, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-18:56:55.843799 140024928073280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732272970,"executesMs":44937,"compactionJobId":8157,"attempt":0,"compactionInputRawBytes":1927529851,"compactionInputZipBytes":553802042,"compactionOutputRawBytes":0,"compactionOutputZipBytes":533227378,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-51958f176fb3ab3c0fb7c8f87581082b-542e1c315e306f63-00"}}, response = 2024/11/22-18:56:55.843885 140024928073280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08157/att-00: olev 3, work 44.938 s, result 6.356 ms, install 1.834 ms, input{raw 1.795 GiB zip 528.147 MiB}