2024/11/23-02:05:12.086281 140024919680576 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00: fork to child time = 0.009581 sec 2024/11/23-02:05:12.087154 140024919680576 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00 2024/11/23-02:05:12.093943 140024919680576 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00 2024/11/23-02:05:12.095080 140024919680576 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-02:05:12.095320 140024919680576 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-02:05:12.144122 140024919680576 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16979/att-00/MANIFEST-206347 2024/11/23-02:05:12.157100 140024919680576 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16979/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 239947, last_sequence is 3723342397, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 239809 2024/11/23-02:05:12.157107 140024919680576 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-02:05:12.157212 140024919680576 [version_set.cc:5721] Creating manifest 239947 2024/11/23-02:05:12.191503 140024919680576 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-02:05:12.215443 140024919680576 [compaction_job.cc:2437] [default] [JOB 16979] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-02:05:12.215463 140024919680576 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 12704 Base level 2, inputs: [239943(27MB)], [239884(35MB) 239885(35MB) 239886(36MB) 239887(35MB) 239888(35MB) 239889(34MB) 239890(34MB) 239906(34MB) 239907(35MB) 239908(35MB) 239909(35MB) 239910(34MB) 239911(36MB) 239912(35MB) 239913(35MB) 239914(35MB)] 2024/11/23-02:05:12.221762 140024919680576 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:05:12.221809 140024919680576 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:05:12.221827 140024919680576 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:05:12.221832 140024919680576 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:05:12.221843 140024919680576 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:05:12.750706 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst) /dev/shm/ToplingTemp/Topling-375661-4ZqMbU 2024/11/23-02:05:13.351354 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst): first pass time = 0.60's, 223.470'MB/sec 2024/11/23-02:05:16.168969 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst): old prealloc_size = 147639500, real_size = 32568912 2024/11/23-02:05:16.169636 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930016 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.32 avg-zval = 32.84 2024/11/23-02:05:16.170554 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:17.482158 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239948.sst): second pass time = 2.82's, 38.418'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 28.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2773.879'MB/sec (index lex order gen) rebuild zvType time = 1.15's, 0.203'MB/sec write SST data time = 0.17's, 221.435'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34933.877'MB/sec zip my value time = 2.82's, unzip length = 0.108'GB zip my value throughput = 38.418'MB/sec zip pipeline throughput = 97.043'MB/sec entries = 930016 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.32 avg-zval = 32.84 usrkeys = 930016 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.17 avg-zval = 32.84 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 910352 bytes = 3300048 } seq expand size = 6372464 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3782 value = 3.5417 dict = 1.00 all = 3.6403 } Zip/UnZip{ index = 0.0964 value = 0.2824 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 0.114550 GB avg = 0.123 KB (by entry num) total key len = 0.026040 GB avg = 0.028 KB total ukey len = 0.018600 GB avg = 0.020 KB total ukey num = 0.000930016 Billion total entry num = 0.000930016 Billion write speed all = 0.002268523 MB/sec (with seq num) write speed all = 0.002148471 MB/sec (without seq num) 2024/11/23-02:05:17.515862 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239948: 930016 keys, 36870076 bytes, temperature: kWarm 2024/11/23-02:05:17.515950 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298717515899, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239948, "file_size": 36870076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3723324775, "table_properties": {"data_size": 30544000, "index_size": 1792256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26040448, "raw_average_key_size": 28, "raw_value_size": 108177416, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930016, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298712, "oldest_key_time": 0, "file_creation_time": 1732298712, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239948, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:17.556315 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst) /dev/shm/ToplingTemp/Topling-375661-CGBsgU 2024/11/23-02:05:18.507850 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst): first pass time = 0.95's, 141.059'MB/sec 2024/11/23-02:05:20.516815 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst): old prealloc_size = 147639500, real_size = 36472256 2024/11/23-02:05:20.517345 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927087 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.20 2024/11/23-02:05:20.517387 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:21.158703 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239949.sst): second pass time = 2.01's, 53.905'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.158'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3511.076'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.449'MB/sec write SST data time = 0.12's, 299.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 39998.371'MB/sec zip my value time = 2.01's, unzip length = 0.108'GB zip my value throughput = 53.905'MB/sec zip pipeline throughput = 53.905'MB/sec entries = 927087 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 usrkeys = 927087 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5820 value = 3.1390 dict = 1.00 all = 3.5954 } Zip/UnZip{ index = 0.0945 value = 0.3186 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 0.222809 GB avg = 0.120 KB (by entry num) total key len = 0.051999 GB avg = 0.028 KB total ukey len = 0.037142 GB avg = 0.020 KB total ukey num = 0.001857103 Billion total entry num = 0.001857103 Billion write speed all = 0.004433958 MB/sec (with seq num) write speed all = 0.004194247 MB/sec (without seq num) 2024/11/23-02:05:21.194656 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239949: 927087 keys, 37330542 bytes, temperature: kWarm 2024/11/23-02:05:21.194713 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298721194683, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239949, "file_size": 37330542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34488112, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958436, "raw_average_key_size": 28, "raw_value_size": 108259386, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927087, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298717, "oldest_key_time": 0, "file_creation_time": 1732298717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239949, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:21.232288 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst) /dev/shm/ToplingTemp/Topling-375661-MjH9pT 2024/11/23-02:05:22.003674 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst): first pass time = 0.77's, 174.003'MB/sec 2024/11/23-02:05:23.667101 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst): old prealloc_size = 147639500, real_size = 36706880 2024/11/23-02:05:23.667654 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924878 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0347 GB avg-val = 117.12 avg-zval = 37.54 2024/11/23-02:05:23.667985 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:24.563767 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239950.sst): second pass time = 1.66's, 65.144'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.037'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3299.448'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.349'MB/sec write SST data time = 0.23's, 165.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29932.377'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.144'MB/sec zip pipeline throughput = 72.937'MB/sec entries = 924878 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 37.54 usrkeys = 924878 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.95 avg-zval = 37.54 TagRS{ kinds = RS_Key0_TagN bytes = 144600 } TagArray{ size = 242138 bytes = 877760 } seq expand size = 1694966 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5134 value = 3.1202 dict = 1.00 all = 3.4782 } Zip/UnZip{ index = 0.0951 value = 0.3205 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 0.332825 GB avg = 0.120 KB (by entry num) total key len = 0.077895 GB avg = 0.028 KB total ukey len = 0.055640 GB avg = 0.020 KB total ukey num = 0.002781981 Billion total entry num = 0.002781981 Billion write speed all = 0.006626512 MB/sec (with seq num) write speed all = 0.006267439 MB/sec (without seq num) 2024/11/23-02:05:24.599417 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239950: 924878 keys, 38588581 bytes, temperature: kWarm 2024/11/23-02:05:24.599777 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298724599455, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239950, "file_size": 38588581, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3702085172, "table_properties": {"data_size": 34716080, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896584, "raw_average_key_size": 28, "raw_value_size": 108321228, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924878, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298721, "oldest_key_time": 0, "file_creation_time": 1732298721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239950, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:24.653942 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst) /dev/shm/ToplingTemp/Topling-375661-L9rmjW 2024/11/23-02:05:25.486969 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst): first pass time = 0.83's, 161.130'MB/sec 2024/11/23-02:05:26.997987 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst): old prealloc_size = 147639500, real_size = 33215120 2024/11/23-02:05:26.998508 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926740 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.83 avg-zval = 33.69 2024/11/23-02:05:26.999287 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:28.089552 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239951.sst): second pass time = 1.51's, 71.714'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3512.714'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.241'MB/sec write SST data time = 0.13's, 291.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33738.780'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.714'MB/sec zip pipeline throughput = 73.892'MB/sec entries = 926740 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.69 usrkeys = 926740 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926740 bytes = 3243600 } seq expand size = 6487180 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5120 value = 3.4679 dict = 1.00 all = 3.5971 } Zip/UnZip{ index = 0.0951 value = 0.2884 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 0.447582 GB avg = 0.121 KB (by entry num) total key len = 0.103844 GB avg = 0.028 KB total ukey len = 0.074174 GB avg = 0.020 KB total ukey num = 0.003708721 Billion total entry num = 0.003708721 Billion write speed all = 0.008896121 MB/sec (with seq num) write speed all = 0.008417460 MB/sec (without seq num) 2024/11/23-02:05:28.123843 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239951: 926740 keys, 37313130 bytes, temperature: kWarm 2024/11/23-02:05:28.123904 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298728123873, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239951, "file_size": 37313130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3723323960, "table_properties": {"data_size": 31220032, "index_size": 1763200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948720, "raw_average_key_size": 28, "raw_value_size": 108269148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298724, "oldest_key_time": 0, "file_creation_time": 1732298724, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239951, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:28.159584 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst) /dev/shm/ToplingTemp/Topling-375661-69n9WU 2024/11/23-02:05:29.008052 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst): first pass time = 0.85's, 158.197'MB/sec 2024/11/23-02:05:30.578046 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst): old prealloc_size = 147639500, real_size = 32883520 2024/11/23-02:05:30.578609 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929864 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.34 avg-zval = 33.21 2024/11/23-02:05:30.579361 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:31.525298 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239952.sst): second pass time = 1.57's, 68.963'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.070'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 291.455'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35715.279'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.963'MB/sec zip pipeline throughput = 68.963'MB/sec entries = 929864 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.34 avg-zval = 33.21 usrkeys = 929864 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.34 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929864 bytes = 3138304 } seq expand size = 6509048 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5112 value = 3.5031 dict = 1.00 all = 3.6399 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 0.562272 GB avg = 0.121 KB (by entry num) total key len = 0.129880 GB avg = 0.028 KB total ukey len = 0.092772 GB avg = 0.020 KB total ukey num = 0.004638585 Billion total entry num = 0.004638585 Billion write speed all = 0.011165839 MB/sec (with seq num) write speed all = 0.010567201 MB/sec (without seq num) 2024/11/23-02:05:31.560192 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239952: 929864 keys, 36874246 bytes, temperature: kWarm 2024/11/23-02:05:31.560252 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298731560221, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239952, "file_size": 36874246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3723323429, "table_properties": {"data_size": 30881584, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036192, "raw_average_key_size": 28, "raw_value_size": 108181632, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929864, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298728, "oldest_key_time": 0, "file_creation_time": 1732298728, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239952, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:31.596072 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst) /dev/shm/ToplingTemp/Topling-375661-oGmJYV 2024/11/23-02:05:32.501227 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst): first pass time = 0.91's, 148.288'MB/sec 2024/11/23-02:05:33.892641 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst): old prealloc_size = 147639500, real_size = 32506368 2024/11/23-02:05:33.893266 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926900 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 116.80 avg-zval = 32.92 2024/11/23-02:05:33.894102 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:35.142689 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239953.sst): second pass time = 1.39's, 77.849'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.25's, 29.376'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2991.092'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.212'MB/sec write SST data time = 0.16's, 235.759'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27799.722'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.849'MB/sec zip pipeline throughput = 77.849'MB/sec entries = 926900 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 32.92 usrkeys = 926900 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926900 bytes = 3360032 } seq expand size = 6488300 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5115 value = 3.5484 dict = 1.00 all = 3.6551 } Zip/UnZip{ index = 0.0951 value = 0.2818 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 0.677025 GB avg = 0.122 KB (by entry num) total key len = 0.155834 GB avg = 0.028 KB total ukey len = 0.111310 GB avg = 0.020 KB total ukey num = 0.005565485 Billion total entry num = 0.005565485 Billion write speed all = 0.013434931 MB/sec (with seq num) write speed all = 0.012716712 MB/sec (without seq num) 2024/11/23-02:05:35.176746 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239953: 926900 keys, 36720436 bytes, temperature: kWarm 2024/11/23-02:05:35.176812 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298735176774, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239953, "file_size": 36720436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3723324092, "table_properties": {"data_size": 30510896, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953200, "raw_average_key_size": 28, "raw_value_size": 108264528, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926900, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298731, "oldest_key_time": 0, "file_creation_time": 1732298731, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239953, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:35.214680 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst) /dev/shm/ToplingTemp/Topling-375661-wP0LuW 2024/11/23-02:05:36.121694 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst): first pass time = 0.91's, 147.983'MB/sec 2024/11/23-02:05:37.446977 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst): old prealloc_size = 147639500, real_size = 32452528 2024/11/23-02:05:37.447506 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925716 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.99 avg-zval = 32.89 2024/11/23-02:05:37.448278 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:38.355220 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239954.sst): second pass time = 1.32's, 81.792'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.240'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3483.401'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 287.847'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32278.688'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.792'MB/sec zip pipeline throughput = 81.792'MB/sec entries = 925716 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 32.89 usrkeys = 925716 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.99 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925716 bytes = 3240016 } seq expand size = 6480012 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4454 value = 3.5568 dict = 1.00 all = 3.6725 } Zip/UnZip{ index = 0.0957 value = 0.2812 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 0.791803 GB avg = 0.122 KB (by entry num) total key len = 0.181754 GB avg = 0.028 KB total ukey len = 0.129824 GB avg = 0.020 KB total ukey num = 0.006491201 Billion total entry num = 0.006491201 Billion write speed all = 0.015703728 MB/sec (with seq num) write speed all = 0.014866089 MB/sec (without seq num) 2024/11/23-02:05:38.389166 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239954: 925716 keys, 36547134 bytes, temperature: kWarm 2024/11/23-02:05:38.389228 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298738389196, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239954, "file_size": 36547134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3723323456, "table_properties": {"data_size": 30448416, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920048, "raw_average_key_size": 28, "raw_value_size": 108297740, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298735, "oldest_key_time": 0, "file_creation_time": 1732298735, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239954, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:38.424385 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst) /dev/shm/ToplingTemp/Topling-375661-ul6lbX 2024/11/23-02:05:39.223553 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst): first pass time = 0.80's, 167.955'MB/sec 2024/11/23-02:05:40.541118 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst): old prealloc_size = 147639500, real_size = 32186368 2024/11/23-02:05:40.541661 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930899 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.18 avg-zval = 32.42 2024/11/23-02:05:40.542390 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:41.461708 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239955.sst): second pass time = 1.32's, 82.127'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.400'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 285.648'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34838.040'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.127'MB/sec zip pipeline throughput = 82.127'MB/sec entries = 930899 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 32.42 usrkeys = 930899 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 32.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930899 bytes = 3141792 } seq expand size = 6516293 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5142 value = 3.5833 dict = 1.00 all = 3.7098 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 0.906472 GB avg = 0.122 KB (by entry num) total key len = 0.207819 GB avg = 0.028 KB total ukey len = 0.148442 GB avg = 0.020 KB total ukey num = 0.007422100 Billion total entry num = 0.007422100 Billion write speed all = 0.017972906 MB/sec (with seq num) write speed all = 0.017015190 MB/sec (without seq num) 2024/11/23-02:05:41.495916 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239955: 930899 keys, 36179362 bytes, temperature: kWarm 2024/11/23-02:05:41.495978 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298741495947, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239955, "file_size": 36179362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3723324231, "table_properties": {"data_size": 30182704, "index_size": 1770752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065172, "raw_average_key_size": 28, "raw_value_size": 108152642, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930899, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298738, "oldest_key_time": 0, "file_creation_time": 1732298738, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239955, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:41.531506 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst) /dev/shm/ToplingTemp/Topling-375661-BhXPgT 2024/11/23-02:05:42.388871 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst): first pass time = 0.86's, 156.554'MB/sec 2024/11/23-02:05:43.741766 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst): old prealloc_size = 147639500, real_size = 33645984 2024/11/23-02:05:43.742296 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924028 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.25 avg-zval = 34.26 2024/11/23-02:05:43.743082 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:44.666428 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239956.sst): second pass time = 1.35's, 80.145'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.937'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3526.637'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 296.005'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 26584.083'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.145'MB/sec zip pipeline throughput = 80.145'MB/sec entries = 924028 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.25 avg-zval = 34.26 usrkeys = 924028 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.25 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924028 bytes = 3349616 } seq expand size = 6468196 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5133 value = 3.4225 dict = 1.00 all = 3.5458 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 1.021285 GB avg = 0.122 KB (by entry num) total key len = 0.233692 GB avg = 0.028 KB total ukey len = 0.166923 GB avg = 0.020 KB total ukey num = 0.008346128 Billion total entry num = 0.008346128 Billion write speed all = 0.020241048 MB/sec (with seq num) write speed all = 0.019164156 MB/sec (without seq num) 2024/11/23-02:05:44.700027 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239956: 924028 keys, 37852622 bytes, temperature: kWarm 2024/11/23-02:05:44.700087 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298744700056, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239956, "file_size": 37852622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3723324310, "table_properties": {"data_size": 31656976, "index_size": 1757824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25872784, "raw_average_key_size": 28, "raw_value_size": 108345020, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924028, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298741, "oldest_key_time": 0, "file_creation_time": 1732298741, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239956, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:44.736711 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst) /dev/shm/ToplingTemp/Topling-375661-9bffyT 2024/11/23-02:05:45.562126 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst): first pass time = 0.83's, 162.614'MB/sec 2024/11/23-02:05:46.921765 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst): old prealloc_size = 147639500, real_size = 33445728 2024/11/23-02:05:46.922296 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926936 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.80 avg-zval = 33.93 2024/11/23-02:05:46.924014 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:47.872517 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239957.sst): second pass time = 1.36's, 79.698'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3514.750'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 294.377'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36437.458'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.698'MB/sec zip pipeline throughput = 79.698'MB/sec entries = 926936 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.93 usrkeys = 926936 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926936 bytes = 3244288 } seq expand size = 6488552 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5165 value = 3.4423 dict = 1.00 all = 3.5749 } Zip/UnZip{ index = 0.0951 value = 0.2905 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 1.136037 GB avg = 0.123 KB (by entry num) total key len = 0.259646 GB avg = 0.028 KB total ukey len = 0.185461 GB avg = 0.020 KB total ukey num = 0.009273064 Billion total entry num = 0.009273064 Billion write speed all = 0.022509284 MB/sec (with seq num) write speed all = 0.021312851 MB/sec (without seq num) 2024/11/23-02:05:47.906055 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239957: 926936 keys, 37544296 bytes, temperature: kWarm 2024/11/23-02:05:47.906117 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298747906085, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239957, "file_size": 37544296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3723324352, "table_properties": {"data_size": 31451024, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954208, "raw_average_key_size": 28, "raw_value_size": 108263552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926936, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298744, "oldest_key_time": 0, "file_creation_time": 1732298744, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239957, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:47.946552 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst) /dev/shm/ToplingTemp/Topling-375661-EZ0PPW 2024/11/23-02:05:48.735657 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst): first pass time = 0.79's, 170.097'MB/sec 2024/11/23-02:05:50.078462 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst): old prealloc_size = 147639500, real_size = 33377552 2024/11/23-02:05:50.079011 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926683 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.84 avg-zval = 33.85 2024/11/23-02:05:50.080517 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:51.026243 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239958.sst): second pass time = 1.34's, 80.693'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.541'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.526'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 294.715'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33536.604'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.693'MB/sec zip pipeline throughput = 80.693'MB/sec entries = 926683 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.84 avg-zval = 33.85 usrkeys = 926683 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.84 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926683 bytes = 3243408 } seq expand size = 6486781 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4480 value = 3.4512 dict = 1.00 all = 3.5815 } Zip/UnZip{ index = 0.0957 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 1.250795 GB avg = 0.123 KB (by entry num) total key len = 0.285593 GB avg = 0.028 KB total ukey len = 0.203995 GB avg = 0.020 KB total ukey num = 0.010199747 Billion total entry num = 0.010199747 Billion write speed all = 0.024777278 MB/sec (with seq num) write speed all = 0.023461350 MB/sec (without seq num) 2024/11/23-02:05:51.063620 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239958: 926683 keys, 37475802 bytes, temperature: kWarm 2024/11/23-02:05:51.063684 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298751063652, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239958, "file_size": 37475802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587898319, "largest_seqno": 3723323520, "table_properties": {"data_size": 31371840, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947124, "raw_average_key_size": 28, "raw_value_size": 108270718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926683, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298747, "oldest_key_time": 0, "file_creation_time": 1732298747, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239958, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:51.098117 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst) /dev/shm/ToplingTemp/Topling-375661-hKBXQT 2024/11/23-02:05:51.984520 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst): first pass time = 0.89's, 151.425'MB/sec 2024/11/23-02:05:53.408952 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst): old prealloc_size = 147639500, real_size = 34517168 2024/11/23-02:05:53.409523 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930551 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.23 avg-zval = 34.93 2024/11/23-02:05:53.409979 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:54.224703 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239959.sst): second pass time = 1.42's, 75.993'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 45.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.367'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 293.412'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34311.935'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.993'MB/sec zip pipeline throughput = 75.993'MB/sec entries = 930551 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.23 avg-zval = 34.93 usrkeys = 930551 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.95 avg-zval = 34.93 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 494401 bytes = 1606816 } seq expand size = 3460807 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4457 value = 3.3278 dict = 1.00 all = 3.6153 } Zip/UnZip{ index = 0.0957 value = 0.3005 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 1.362418 GB avg = 0.122 KB (by entry num) total key len = 0.311648 GB avg = 0.028 KB total ukey len = 0.222606 GB avg = 0.020 KB total ukey num = 0.011130298 Billion total entry num = 0.011130298 Billion write speed all = 0.026996224 MB/sec (with seq num) write speed all = 0.025560314 MB/sec (without seq num) 2024/11/23-02:05:54.258695 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239959: 930551 keys, 37125045 bytes, temperature: kWarm 2024/11/23-02:05:54.258763 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298754258726, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239959, "file_size": 37125045, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3723324266, "table_properties": {"data_size": 32502624, "index_size": 1781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055428, "raw_average_key_size": 28, "raw_value_size": 108162342, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930551, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298751, "oldest_key_time": 0, "file_creation_time": 1732298751, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239959, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:54.295923 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst) /dev/shm/ToplingTemp/Topling-375661-oUStLT 2024/11/23-02:05:55.167862 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst): first pass time = 0.87's, 153.937'MB/sec 2024/11/23-02:05:56.558244 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst): old prealloc_size = 147639500, real_size = 35492064 2024/11/23-02:05:56.558877 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080466 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.22 avg-zval = 30.67 2024/11/23-02:05:56.559105 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:05:57.202191 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239960.sst): second pass time = 1.39's, 74.811'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.337'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.537'MB/sec write SST data time = 0.14's, 262.006'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.029'MB, throughput = 32041.170'MB/sec zip my value time = 1.39's, unzip length = 0.104'GB zip my value throughput = 74.811'MB/sec zip pipeline throughput = 74.811'MB/sec entries = 1080466 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.22 avg-zval = 30.67 usrkeys = 1080466 avg-key = 20.00 avg-zkey = 1.93 avg-val = 97.05 avg-zval = 30.67 TagRS{ kinds = RS_Key0_TagN bytes = 168920 } TagArray{ size = 128213 bytes = 432736 } seq expand size = 897491 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3789 value = 3.1372 dict = 1.00 all = 3.6409 } Zip/UnZip{ index = 0.0963 value = 0.3188 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 1.467280 GB avg = 0.120 KB (by entry num) total key len = 0.341901 GB avg = 0.028 KB total ukey len = 0.244215 GB avg = 0.020 KB total ukey num = 0.012210764 Billion total entry num = 0.012210764 Billion write speed all = 0.029173711 MB/sec (with seq num) write speed all = 0.027598487 MB/sec (without seq num) 2024/11/23-02:05:57.235243 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239960: 1080466 keys, 36863449 bytes, temperature: kWarm 2024/11/23-02:05:57.235304 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298757235273, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239960, "file_size": 36863449, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3406054210, "table_properties": {"data_size": 33139728, "index_size": 2082048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30253048, "raw_average_key_size": 28, "raw_value_size": 103964769, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080466, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298754, "oldest_key_time": 0, "file_creation_time": 1732298754, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239960, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:05:57.277437 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst) /dev/shm/ToplingTemp/Topling-375661-8sYNmX 2024/11/23-02:05:58.174537 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst): first pass time = 0.90's, 149.619'MB/sec 2024/11/23-02:05:59.504830 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst): old prealloc_size = 147639500, real_size = 32574704 2024/11/23-02:05:59.505353 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926809 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.82 avg-zval = 32.99 2024/11/23-02:05:59.506139 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:06:00.410816 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239961.sst): second pass time = 1.33's, 81.423'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.608'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3515.372'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 288.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34010.326'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.423'MB/sec zip pipeline throughput = 81.423'MB/sec entries = 926809 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 32.99 usrkeys = 926809 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 32.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926809 bytes = 3359696 } seq expand size = 6487663 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5120 value = 3.5405 dict = 1.00 all = 3.6483 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 1.582035 GB avg = 0.120 KB (by entry num) total key len = 0.367852 GB avg = 0.028 KB total ukey len = 0.262751 GB avg = 0.020 KB total ukey num = 0.013137573 Billion total entry num = 0.013137573 Billion write speed all = 0.031441010 MB/sec (with seq num) write speed all = 0.029746313 MB/sec (without seq num) 2024/11/23-02:06:00.441865 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239961: 926809 keys, 36788830 bytes, temperature: kWarm 2024/11/23-02:06:00.441926 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298760441895, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239961, "file_size": 36788830, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3406054211, "largest_seqno": 3723324432, "table_properties": {"data_size": 30579488, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950652, "raw_average_key_size": 28, "raw_value_size": 108267094, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298757, "oldest_key_time": 0, "file_creation_time": 1732298757, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239961, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:06:00.477607 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst) /dev/shm/ToplingTemp/Topling-375661-Nr0oaU 2024/11/23-02:06:01.297214 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst): first pass time = 0.82's, 163.766'MB/sec 2024/11/23-02:06:02.642084 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst): old prealloc_size = 147639500, real_size = 32741008 2024/11/23-02:06:02.642654 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930655 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.22 avg-zval = 33.03 2024/11/23-02:06:02.644396 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:06:03.572615 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239962.sst): second pass time = 1.34's, 80.463'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.727'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.791'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 285.372'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31125.256'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.463'MB/sec zip pipeline throughput = 80.463'MB/sec entries = 930655 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 33.03 usrkeys = 930655 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930655 bytes = 3373632 } seq expand size = 6514585 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5122 value = 3.5188 dict = 1.00 all = 3.6308 } Zip/UnZip{ index = 0.0951 value = 0.2842 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 1.696709 GB avg = 0.121 KB (by entry num) total key len = 0.393910 GB avg = 0.028 KB total ukey len = 0.281365 GB avg = 0.020 KB total ukey num = 0.014068228 Billion total entry num = 0.014068228 Billion write speed all = 0.033708535 MB/sec (with seq num) write speed all = 0.031893879 MB/sec (without seq num) 2024/11/23-02:06:03.606392 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239962: 930655 keys, 36966034 bytes, temperature: kWarm 2024/11/23-02:06:03.606457 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298763606424, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239962, "file_size": 36966034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3723324630, "table_properties": {"data_size": 30737536, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058340, "raw_average_key_size": 28, "raw_value_size": 108159446, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930655, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298760, "oldest_key_time": 0, "file_creation_time": 1732298760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239962, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:06:03.652205 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst) /dev/shm/ToplingTemp/Topling-375661-1Chu7T 2024/11/23-02:06:04.460824 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst): first pass time = 0.81's, 165.992'MB/sec 2024/11/23-02:06:05.807927 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst): old prealloc_size = 147639500, real_size = 33102096 2024/11/23-02:06:05.808498 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929307 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.43 avg-zval = 33.45 2024/11/23-02:06:05.810181 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:06:06.723594 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239963.sst): second pass time = 1.35's, 80.357'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.762'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3243.515'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 289.958'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33354.681'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.357'MB/sec zip pipeline throughput = 80.357'MB/sec entries = 929307 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.43 avg-zval = 33.45 usrkeys = 929307 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.43 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929307 bytes = 3368752 } seq expand size = 6505149 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4426 value = 3.4802 dict = 1.00 all = 3.5961 } Zip/UnZip{ index = 0.0958 value = 0.2873 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.811411 GB avg = 0.121 KB (by entry num) total key len = 0.419931 GB avg = 0.028 KB total ukey len = 0.299951 GB avg = 0.020 KB total ukey num = 0.014997535 Billion total entry num = 0.014997535 Billion write speed all = 0.035975683 MB/sec (with seq num) write speed all = 0.034041255 MB/sec (without seq num) 2024/11/23-02:06:06.759001 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239963: 929307 keys, 37323428 bytes, temperature: kWarm 2024/11/23-02:06:06.759064 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298766759032, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239963, "file_size": 37323428, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3723324479, "table_properties": {"data_size": 31089728, "index_size": 1779840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020596, "raw_average_key_size": 28, "raw_value_size": 108197250, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929307, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298763, "oldest_key_time": 0, "file_creation_time": 1732298763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239963, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:06:06.800587 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst) /dev/shm/ToplingTemp/Topling-375661-FSqufT 2024/11/23-02:06:06.889474 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst): first pass time = 0.09's, 238.187'MB/sec 2024/11/23-02:06:07.076344 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst): old prealloc_size = 147639500, real_size = 6127232 2024/11/23-02:06:07.076479 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 146010 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0171 GB zip-val = 0.0058 GB avg-val = 116.94 avg-zval = 39.79 2024/11/23-02:06:07.076618 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/23-02:06:07.214647 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00/239964.sst): second pass time = 0.19's, 91.660'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 48.787'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2532.939'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.310'MB/sec write SST data time = 0.02's, 330.220'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.180'MB, throughput = 6778.825'MB/sec zip my value time = 0.19's, unzip length = 0.017'GB zip my value throughput = 91.660'MB/sec zip pipeline throughput = 91.660'MB/sec entries = 146010 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.94 avg-zval = 39.79 usrkeys = 146010 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.94 avg-zval = 39.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 146010 bytes = 474544 } seq expand size = 1022070 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0171 GB dict = 0.18 MB all = 0.0212 GB } __ZipSize{ index = 0.0003 GB value = 0.0058 GB dict = 0.18 MB all = 0.0067 GB } UnZip/Zip{ index = 10.3984 value = 2.9389 dict = 1.00 all = 3.1368 } Zip/UnZip{ index = 0.0962 value = 0.3403 dict = 1.00 all = 0.3188 } ---------------------------- total value len = 1.829508 GB avg = 0.121 KB (by entry num) total key len = 0.424019 GB avg = 0.028 KB total ukey len = 0.302871 GB avg = 0.020 KB total ukey num = 0.015143545 Billion total entry num = 0.015143545 Billion write speed all = 0.036333078 MB/sec (with seq num) write speed all = 0.034379832 MB/sec (without seq num) 2024/11/23-02:06:07.226179 140024919680576 [compaction_job.cc:1948] [default] [JOB 16979] Generated table #239964: 146010 keys, 6746523 bytes, temperature: kWarm 2024/11/23-02:06:07.226241 140024919680576 EVENT_LOG_v1 {"time_micros": 1732298767226203, "cf_name": "default", "job": 16979, "event": "table_file_creation", "file_number": 239964, "file_size": 6746523, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3678505207, "largest_seqno": 3721834581, "table_properties": {"data_size": 5809712, "index_size": 280832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4088280, "raw_average_key_size": 28, "raw_value_size": 17074420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 146010, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732298766, "oldest_key_time": 0, "file_creation_time": 1732298766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 239964, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:06:07.238940 140024919680576 [compaction_job.cc:703] job-16979: subcompact[0], size: 0.601110 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16979/att-00 [239948,239949,239950,239951,239952,239953,239954,239955,239956,239957,239958,239959,239960,239961,239962,239963,239964] 2024/11/23-02:06:07.346744 140024919680576 (Original Log Time 2024/11/23-02:05:12.215513) EVENT_LOG_v1 {"time_micros": 1732298712215475, "job": 16979, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [239943], "files_L3": [239884, 239885, 239886, 239887, 239888, 239889, 239890, 239906, 239907, 239908, 239909, 239910, 239911, 239912, 239913, 239914], "score": 0.988515, "input_data_size": 622603832, "oldest_snapshot_seqno": 3723342239} 2024/11/23-02:06:07.346748 140024919680576 (Original Log Time 2024/11/23-02:06:07.345913) [compaction_job.cc:2022] [default] [JOB 16979] Compacted 1@2 + 16@3 files to L3 => 601109536 bytes 2024/11/23-02:06:07.346749 140024919680576 (Original Log Time 2024/11/23-02:06:07.346678) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 11.3 rd, 10.9 wr, level 3, files in(1, 16) out(17 +0 blob) MB in(27.1, 566.7 +0.0 blob) out(573.3 +0.0 blob), read-write-amplify(43.1) write-amplify(21.2) OK, records in: 15230524, records dropped: 86979 output_compression: Snappy 2024/11/23-02:06:07.346755 140024919680576 (Original Log Time 2024/11/23-02:06:07.346726) EVENT_LOG_v1 {"time_micros": 1732298767346701, "cf": "default", "job": 16979, "event": "compaction_finished", "compaction_time_micros": 55023434, "compaction_time_cpu_micros": 23879669, "output_level": 3, "num_output_files": 17, "total_output_size": 601109536, "num_input_records": 15230524, "num_output_records": 15143545, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 387577223, "file_range_sync_nanos": 0, "file_fsync_nanos": 16393, "file_prepare_write_nanos": 39115, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/23-02:06:07.483070 140024919680576 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732298712,"executesMs":55253,"compactionJobId":16979,"attempt":0,"compactionInputRawBytes":2180998609,"compactionInputZipBytes":622603832,"compactionOutputRawBytes":0,"compactionOutputZipBytes":601109536,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a72bf9b2ac074bd0861cb9d0f0342c5e-68c6bc733e672b4e-00"}}, response = 2024/11/23-02:06:07.483138 140024919680576 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16979/att-00: olev 3, work 55.253 s, result 6.324 ms, install 0.862 ms, input{raw 2.031 GiB zip 593.761 MiB}