2024/11/23-07:20:29.938661 140024760219200 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00: fork to child time = 0.009188 sec 2024/11/23-07:20:29.939422 140024760219200 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00 2024/11/23-07:20:29.947654 140024760219200 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00 2024/11/23-07:20:29.948895 140024760219200 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-07:20:29.949172 140024760219200 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-07:20:30.000931 140024760219200 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21761/att-00/MANIFEST-206347 2024/11/23-07:20:30.037275 140024760219200 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21761/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 250274, last_sequence is 3857171320, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 250111 2024/11/23-07:20:30.037284 140024760219200 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-07:20:30.037403 140024760219200 [version_set.cc:5721] Creating manifest 250274 2024/11/23-07:20:30.073901 140024760219200 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-07:20:30.092185 140024760219200 [compaction_job.cc:2437] [default] [JOB 21761] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-07:20:30.092206 140024760219200 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 16322 Base level 2, inputs: [250207(27MB)], [250259(35MB) 250260(7459KB) 250151(35MB) 250168(35MB) 250169(35MB) 250170(35MB) 250171(36MB) 250172(34MB) 250173(36MB) 250174(35MB) 250175(35MB) 250176(35MB) 250177(35MB) 250178(35MB) 250179(35MB) 250261(35MB)] 2024/11/23-07:20:30.099500 140024760219200 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:20:30.099532 140024760219200 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:20:30.099552 140024760219200 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-07:20:30.099557 140024760219200 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:20:30.099569 140024760219200 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-07:20:30.677996 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst) /dev/shm/ToplingTemp/Topling-445932-7tj4Cs 2024/11/23-07:20:31.226252 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst): first pass time = 0.55's, 244.828'MB/sec 2024/11/23-07:20:32.555666 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst): old prealloc_size = 147639500, real_size = 33384864 2024/11/23-07:20:32.556222 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930105 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.30 avg-zval = 33.74 2024/11/23-07:20:32.557026 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:33.486368 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250275.sst): second pass time = 1.33's, 81.422'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.284'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3316.663'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 290.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33141.453'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.422'MB/sec zip pipeline throughput = 81.422'MB/sec entries = 930105 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.30 avg-zval = 33.74 usrkeys = 930105 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 33.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930105 bytes = 3255376 } seq expand size = 6510735 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5124 value = 3.4470 dict = 1.00 all = 3.5799 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.114686 GB avg = 0.123 KB (by entry num) total key len = 0.026043 GB avg = 0.028 KB total ukey len = 0.018602 GB avg = 0.020 KB total ukey num = 0.000930105 Billion total entry num = 0.000930105 Billion write speed all = 0.001739744 MB/sec (with seq num) write speed all = 0.001647757 MB/sec (without seq num) 2024/11/23-07:20:33.521948 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250275: 930105 keys, 37492440 bytes, temperature: kWarm 2024/11/23-07:20:33.522044 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317633521987, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250275, "file_size": 37492440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3856846038, "table_properties": {"data_size": 31382608, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26042940, "raw_average_key_size": 28, "raw_value_size": 108174906, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930105, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317630, "oldest_key_time": 0, "file_creation_time": 1732317630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250275, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:33.558509 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst) /dev/shm/ToplingTemp/Topling-445932-51QKzv 2024/11/23-07:20:34.550744 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst): first pass time = 0.99's, 135.273'MB/sec 2024/11/23-07:20:35.907440 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst): old prealloc_size = 147639500, real_size = 32914064 2024/11/23-07:20:35.908009 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929037 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.47 avg-zval = 33.25 2024/11/23-07:20:35.908807 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:36.839430 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250276.sst): second pass time = 1.36's, 79.800'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.622'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 288.521'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31310.475'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.800'MB/sec zip pipeline throughput = 79.800'MB/sec entries = 929037 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.47 avg-zval = 33.25 usrkeys = 929037 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.47 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929037 bytes = 3367776 } seq expand size = 6503259 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.3769 value = 3.5028 dict = 1.00 all = 3.6144 } Zip/UnZip{ index = 0.0964 value = 0.2855 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 0.229394 GB avg = 0.123 KB (by entry num) total key len = 0.052056 GB avg = 0.028 KB total ukey len = 0.037183 GB avg = 0.020 KB total ukey num = 0.001859142 Billion total entry num = 0.001859142 Billion write speed all = 0.003479250 MB/sec (with seq num) write speed all = 0.003295390 MB/sec (without seq num) 2024/11/23-07:20:36.879133 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250276: 929037 keys, 37134678 bytes, temperature: kWarm 2024/11/23-07:20:36.879200 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317636879166, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250276, "file_size": 37134678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3856844858, "table_properties": {"data_size": 30891008, "index_size": 1790592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013036, "raw_average_key_size": 28, "raw_value_size": 108204706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929037, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317633, "oldest_key_time": 0, "file_creation_time": 1732317633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250276, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:36.917731 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst) /dev/shm/ToplingTemp/Topling-445932-x2NSCt 2024/11/23-07:20:37.780257 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst): first pass time = 0.86's, 155.617'MB/sec 2024/11/23-07:20:39.098752 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst): old prealloc_size = 147639500, real_size = 32798160 2024/11/23-07:20:39.099326 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929037 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.47 avg-zval = 33.15 2024/11/23-07:20:39.100100 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:40.022972 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250277.sst): second pass time = 1.32's, 82.113'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3265.433'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 289.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32319.509'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.113'MB/sec zip pipeline throughput = 82.113'MB/sec entries = 929037 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 33.15 usrkeys = 929037 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929037 bytes = 3251648 } seq expand size = 6503259 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5133 value = 3.5133 dict = 1.00 all = 3.6371 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 0.344102 GB avg = 0.123 KB (by entry num) total key len = 0.078069 GB avg = 0.028 KB total ukey len = 0.055764 GB avg = 0.020 KB total ukey num = 0.002788179 Billion total entry num = 0.002788179 Billion write speed all = 0.005218624 MB/sec (with seq num) write speed all = 0.004942898 MB/sec (without seq num) 2024/11/23-07:20:40.058687 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250277: 929037 keys, 36902686 bytes, temperature: kWarm 2024/11/23-07:20:40.058751 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317640058719, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250277, "file_size": 36902686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3856846446, "table_properties": {"data_size": 30798336, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013036, "raw_average_key_size": 28, "raw_value_size": 108204774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929037, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317636, "oldest_key_time": 0, "file_creation_time": 1732317636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:40.092604 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst) /dev/shm/ToplingTemp/Topling-445932-sGM2Xr 2024/11/23-07:20:40.893992 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst): first pass time = 0.80's, 167.490'MB/sec 2024/11/23-07:20:42.207851 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst): old prealloc_size = 147639500, real_size = 33199008 2024/11/23-07:20:42.208389 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928818 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.50 avg-zval = 33.59 2024/11/23-07:20:42.209193 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:43.160670 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250278.sst): second pass time = 1.31's, 82.406'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.273'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.883'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 291.365'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31998.222'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.406'MB/sec zip pipeline throughput = 82.406'MB/sec entries = 928818 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 33.59 usrkeys = 928818 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928818 bytes = 3366976 } seq expand size = 6501726 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5131 value = 3.4683 dict = 1.00 all = 3.5869 } Zip/UnZip{ index = 0.0951 value = 0.2883 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.458814 GB avg = 0.123 KB (by entry num) total key len = 0.104076 GB avg = 0.028 KB total ukey len = 0.074340 GB avg = 0.020 KB total ukey num = 0.003716997 Billion total entry num = 0.003716997 Billion write speed all = 0.006957845 MB/sec (with seq num) write speed all = 0.006590281 MB/sec (without seq num) 2024/11/23-07:20:43.198974 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250278: 928818 keys, 37419348 bytes, temperature: kWarm 2024/11/23-07:20:43.199036 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317643199004, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250278, "file_size": 37419348, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3856845432, "table_properties": {"data_size": 31199632, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006904, "raw_average_key_size": 28, "raw_value_size": 108210824, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928818, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317640, "oldest_key_time": 0, "file_creation_time": 1732317640, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250278, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:43.242625 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst) /dev/shm/ToplingTemp/Topling-445932-LOF3ct 2024/11/23-07:20:44.084226 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst): first pass time = 0.84's, 159.486'MB/sec 2024/11/23-07:20:45.411948 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst): old prealloc_size = 147639500, real_size = 32335856 2024/11/23-07:20:45.412484 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928814 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.50 avg-zval = 32.66 2024/11/23-07:20:45.413285 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:46.318098 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250279.sst): second pass time = 1.33's, 81.545'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3423.547'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 283.631'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34329.211'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.545'MB/sec zip pipeline throughput = 81.545'MB/sec entries = 928814 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 32.66 usrkeys = 928814 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 32.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928814 bytes = 3366960 } seq expand size = 6501698 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5127 value = 3.5670 dict = 1.00 all = 3.6716 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 0.573527 GB avg = 0.123 KB (by entry num) total key len = 0.130083 GB avg = 0.028 KB total ukey len = 0.092916 GB avg = 0.020 KB total ukey num = 0.004645811 Billion total entry num = 0.004645811 Billion write speed all = 0.008696929 MB/sec (with seq num) write speed all = 0.008237534 MB/sec (without seq num) 2024/11/23-07:20:46.356194 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250279: 928814 keys, 36556120 bytes, temperature: kWarm 2024/11/23-07:20:46.356279 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317646356227, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250279, "file_size": 36556120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3856845680, "table_properties": {"data_size": 30336416, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006792, "raw_average_key_size": 28, "raw_value_size": 108210948, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928814, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317643, "oldest_key_time": 0, "file_creation_time": 1732317643, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250279, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:46.397617 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst) /dev/shm/ToplingTemp/Topling-445932-RK08Nt 2024/11/23-07:20:47.228545 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst): first pass time = 0.83's, 161.536'MB/sec 2024/11/23-07:20:48.593073 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst): old prealloc_size = 147639500, real_size = 34075104 2024/11/23-07:20:48.593623 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930119 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.30 avg-zval = 34.48 2024/11/23-07:20:48.594395 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:49.515135 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250280.sst): second pass time = 1.36's, 79.347'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3369.146'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 297.974'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35943.885'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.347'MB/sec zip pipeline throughput = 79.347'MB/sec entries = 930119 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.30 avg-zval = 34.48 usrkeys = 930119 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 34.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930119 bytes = 3139168 } seq expand size = 6510833 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5126 value = 3.3728 dict = 1.00 all = 3.5259 } Zip/UnZip{ index = 0.0951 value = 0.2965 dict = 1.00 all = 0.2836 } ---------------------------- total value len = 0.688212 GB avg = 0.123 KB (by entry num) total key len = 0.156126 GB avg = 0.028 KB total ukey len = 0.111519 GB avg = 0.020 KB total ukey num = 0.005575930 Billion total entry num = 0.005575930 Billion write speed all = 0.010435985 MB/sec (with seq num) write speed all = 0.009884638 MB/sec (without seq num) 2024/11/23-07:20:49.553818 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250280: 930119 keys, 38066212 bytes, temperature: kWarm 2024/11/23-07:20:49.553882 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317649553850, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250280, "file_size": 38066212, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3856846564, "table_properties": {"data_size": 32072848, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26043332, "raw_average_key_size": 28, "raw_value_size": 108174430, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930119, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317646, "oldest_key_time": 0, "file_creation_time": 1732317646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250280, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:49.589838 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst) /dev/shm/ToplingTemp/Topling-445932-lnwVpu 2024/11/23-07:20:50.360599 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst): first pass time = 0.77's, 174.145'MB/sec 2024/11/23-07:20:51.744195 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst): old prealloc_size = 147639500, real_size = 34326368 2024/11/23-07:20:51.744798 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932754 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0323 GB avg-val = 115.89 avg-zval = 34.64 2024/11/23-07:20:51.746262 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:52.555916 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250281.sst): second pass time = 1.38's, 78.224'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 45.449'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3075.411'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 283.948'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33887.464'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.224'MB/sec zip pipeline throughput = 78.224'MB/sec entries = 932754 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.89 avg-zval = 34.64 usrkeys = 932754 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.51 avg-zval = 34.64 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 481305 bytes = 1564256 } seq expand size = 3369135 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4449 value = 3.3460 dict = 1.00 all = 3.6382 } Zip/UnZip{ index = 0.0957 value = 0.2989 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 0.799682 GB avg = 0.123 KB (by entry num) total key len = 0.182243 GB avg = 0.028 KB total ukey len = 0.130174 GB avg = 0.020 KB total ukey num = 0.006508684 Billion total entry num = 0.006508684 Billion write speed all = 0.012136097 MB/sec (with seq num) write speed all = 0.011492545 MB/sec (without seq num) 2024/11/23-07:20:52.591162 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250281: 932754 keys, 36890907 bytes, temperature: kWarm 2024/11/23-07:20:52.591226 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317652591195, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250281, "file_size": 36890907, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3856844463, "table_properties": {"data_size": 32306960, "index_size": 1786048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26117112, "raw_average_key_size": 28, "raw_value_size": 108100696, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932754, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317649, "oldest_key_time": 0, "file_creation_time": 1732317649, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250281, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:52.625523 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst) /dev/shm/ToplingTemp/Topling-445932-4glxJu 2024/11/23-07:20:53.482133 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst): first pass time = 0.86's, 156.692'MB/sec 2024/11/23-07:20:54.844459 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst): old prealloc_size = 147639500, real_size = 35489312 2024/11/23-07:20:54.845078 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080418 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.23 avg-zval = 30.67 2024/11/23-07:20:54.845356 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:55.497068 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250282.sst): second pass time = 1.36's, 76.358'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 56.586'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3479.328'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.527'MB/sec write SST data time = 0.14's, 264.375'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.026'MB, throughput = 31362.833'MB/sec zip my value time = 1.36's, unzip length = 0.104'GB zip my value throughput = 76.358'MB/sec zip pipeline throughput = 76.358'MB/sec entries = 1080418 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.23 avg-zval = 30.67 usrkeys = 1080418 avg-key = 20.00 avg-zkey = 1.93 avg-val = 97.16 avg-zval = 30.67 TagRS{ kinds = RS_Key0_TagN bytes = 168920 } TagArray{ size = 143464 bytes = 502144 } seq expand size = 1004248 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3794 value = 3.1374 dict = 1.00 all = 3.6346 } Zip/UnZip{ index = 0.0963 value = 0.3187 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 0.904652 GB avg = 0.119 KB (by entry num) total key len = 0.212495 GB avg = 0.028 KB total ukey len = 0.151782 GB avg = 0.020 KB total ukey num = 0.007589102 Billion total entry num = 0.007589102 Billion write speed all = 0.013806871 MB/sec (with seq num) write speed all = 0.013056518 MB/sec (without seq num) 2024/11/23-07:20:55.537623 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250282: 1080418 keys, 36927421 bytes, temperature: kWarm 2024/11/23-07:20:55.537690 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317655537657, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250282, "file_size": 36927421, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3408995575, "table_properties": {"data_size": 33137168, "index_size": 2081856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30251704, "raw_average_key_size": 28, "raw_value_size": 103966029, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080418, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317652, "oldest_key_time": 0, "file_creation_time": 1732317652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:55.575568 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst) /dev/shm/ToplingTemp/Topling-445932-eHrKVt 2024/11/23-07:20:56.496108 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst): first pass time = 0.92's, 145.809'MB/sec 2024/11/23-07:20:57.813944 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst): old prealloc_size = 147639500, real_size = 31677952 2024/11/23-07:20:57.814491 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928770 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0297 GB avg-val = 116.51 avg-zval = 31.95 2024/11/23-07:20:57.816205 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:20:58.693274 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250283.sst): second pass time = 1.32's, 82.159'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.825'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.473'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 280.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34928.856'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.159'MB/sec zip pipeline throughput = 82.159'MB/sec entries = 928770 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 31.95 usrkeys = 928770 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928770 bytes = 3366800 } seq expand size = 6501390 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5122 value = 3.6461 dict = 1.00 all = 3.7389 } Zip/UnZip{ index = 0.0951 value = 0.2743 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 1.019366 GB avg = 0.120 KB (by entry num) total key len = 0.238500 GB avg = 0.028 KB total ukey len = 0.170357 GB avg = 0.020 KB total ukey num = 0.008517872 Billion total entry num = 0.008517872 Billion write speed all = 0.015545411 MB/sec (with seq num) write speed all = 0.014703263 MB/sec (without seq num) 2024/11/23-07:20:58.728555 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250283: 928770 keys, 35898000 bytes, temperature: kWarm 2024/11/23-07:20:58.728617 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317658728586, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250283, "file_size": 35898000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3408995576, "largest_seqno": 3856846937, "table_properties": {"data_size": 29678512, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005560, "raw_average_key_size": 28, "raw_value_size": 108212236, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928770, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317655, "oldest_key_time": 0, "file_creation_time": 1732317655, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:20:58.816125 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst) /dev/shm/ToplingTemp/Topling-445932-ps5UIu 2024/11/23-07:20:59.655159 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst): first pass time = 0.84's, 159.975'MB/sec 2024/11/23-07:21:00.966335 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst): old prealloc_size = 147639500, real_size = 32808784 2024/11/23-07:21:00.966928 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931456 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.09 avg-zval = 33.05 2024/11/23-07:21:00.967688 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:01.919028 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250284.sst): second pass time = 1.31's, 82.513'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.510'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3144.058'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 286.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37747.410'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.513'MB/sec zip pipeline throughput = 82.513'MB/sec entries = 931456 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.09 avg-zval = 33.05 usrkeys = 931456 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.09 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931456 bytes = 3027248 } seq expand size = 6520192 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.3750 value = 3.5132 dict = 1.00 all = 3.6584 } Zip/UnZip{ index = 0.0964 value = 0.2846 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 1.134023 GB avg = 0.120 KB (by entry num) total key len = 0.264581 GB avg = 0.028 KB total ukey len = 0.188987 GB avg = 0.020 KB total ukey num = 0.009449328 Billion total entry num = 0.009449328 Billion write speed all = 0.017284040 MB/sec (with seq num) write speed all = 0.016349837 MB/sec (without seq num) 2024/11/23-07:21:01.956192 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250284: 931456 keys, 36687344 bytes, temperature: kWarm 2024/11/23-07:21:01.956271 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317661956227, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250284, "file_size": 36687344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3795922131, "largest_seqno": 3856846964, "table_properties": {"data_size": 30780160, "index_size": 1795584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080768, "raw_average_key_size": 28, "raw_value_size": 108136996, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931456, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317658, "oldest_key_time": 0, "file_creation_time": 1732317658, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:01.993165 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst) /dev/shm/ToplingTemp/Topling-445932-DE5CSv 2024/11/23-07:21:02.852764 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst): first pass time = 0.86's, 156.148'MB/sec 2024/11/23-07:21:04.159112 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst): old prealloc_size = 147639500, real_size = 31890208 2024/11/23-07:21:04.159664 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933606 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 115.76 avg-zval = 32.01 2024/11/23-07:21:04.161433 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:05.079766 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250285.sst): second pass time = 1.31's, 82.774'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.366'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3370.198'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 279.722'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33348.100'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.774'MB/sec zip pipeline throughput = 82.774'MB/sec entries = 933606 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.76 avg-zval = 32.01 usrkeys = 933606 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.76 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933606 bytes = 3501040 } seq expand size = 6535242 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5121 value = 3.6170 dict = 1.00 all = 3.7034 } Zip/UnZip{ index = 0.0951 value = 0.2765 dict = 1.00 all = 0.2700 } ---------------------------- total value len = 1.248635 GB avg = 0.120 KB (by entry num) total key len = 0.290722 GB avg = 0.028 KB total ukey len = 0.207659 GB avg = 0.020 KB total ukey num = 0.010382934 Billion total entry num = 0.010382934 Billion write speed all = 0.019022732 MB/sec (with seq num) write speed all = 0.017996268 MB/sec (without seq num) 2024/11/23-07:21:05.115585 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250285: 933606 keys, 36241608 bytes, temperature: kWarm 2024/11/23-07:21:05.115648 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317665115614, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250285, "file_size": 36241608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3856846212, "table_properties": {"data_size": 29880336, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26140968, "raw_average_key_size": 28, "raw_value_size": 108076792, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933606, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317661, "oldest_key_time": 0, "file_creation_time": 1732317661, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:05.152807 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst) /dev/shm/ToplingTemp/Topling-445932-CgG7Tt 2024/11/23-07:21:06.063847 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst): first pass time = 0.91's, 147.330'MB/sec 2024/11/23-07:21:07.392824 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst): old prealloc_size = 147639500, real_size = 33981280 2024/11/23-07:21:07.393375 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924547 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.17 avg-zval = 34.59 2024/11/23-07:21:07.395077 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:08.331737 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250286.sst): second pass time = 1.33's, 81.554'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.674'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.190'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 295.080'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35571.692'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.554'MB/sec zip pipeline throughput = 81.554'MB/sec entries = 924547 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.17 avg-zval = 34.59 usrkeys = 924547 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.17 avg-zval = 34.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924547 bytes = 3351504 } seq expand size = 6471829 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4436 value = 3.3875 dict = 1.00 all = 3.5146 } Zip/UnZip{ index = 0.0958 value = 0.2952 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 1.363437 GB avg = 0.121 KB (by entry num) total key len = 0.316609 GB avg = 0.028 KB total ukey len = 0.226150 GB avg = 0.020 KB total ukey num = 0.011307481 Billion total entry num = 0.011307481 Billion write speed all = 0.020760481 MB/sec (with seq num) write speed all = 0.019642661 MB/sec (without seq num) 2024/11/23-07:21:08.371472 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250286: 924547 keys, 38189008 bytes, temperature: kWarm 2024/11/23-07:21:08.371536 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317668371504, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250286, "file_size": 38189008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3856846535, "table_properties": {"data_size": 31979408, "index_size": 1770560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887316, "raw_average_key_size": 28, "raw_value_size": 108330482, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924547, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317665, "oldest_key_time": 0, "file_creation_time": 1732317665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:08.408135 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst) /dev/shm/ToplingTemp/Topling-445932-Ub6fYr 2024/11/23-07:21:09.265787 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst): first pass time = 0.86's, 156.506'MB/sec 2024/11/23-07:21:10.576915 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst): old prealloc_size = 147639500, real_size = 33031840 2024/11/23-07:21:10.577496 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928937 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.49 avg-zval = 33.39 2024/11/23-07:21:10.579176 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:11.515742 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250287.sst): second pass time = 1.31's, 82.597'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.544'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 290.097'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34375.151'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.597'MB/sec zip pipeline throughput = 82.597'MB/sec entries = 928937 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.49 avg-zval = 33.39 usrkeys = 928937 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.49 avg-zval = 33.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928937 bytes = 3367408 } seq expand size = 6502559 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4433 value = 3.4883 dict = 1.00 all = 3.6029 } Zip/UnZip{ index = 0.0958 value = 0.2867 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 1.478147 GB avg = 0.121 KB (by entry num) total key len = 0.342620 GB avg = 0.028 KB total ukey len = 0.244728 GB avg = 0.020 KB total ukey num = 0.012236418 Billion total entry num = 0.012236418 Billion write speed all = 0.022498489 MB/sec (with seq num) write speed all = 0.021288885 MB/sec (without seq num) 2024/11/23-07:21:11.552848 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250287: 928937 keys, 37252402 bytes, temperature: kWarm 2024/11/23-07:21:11.552912 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317671552880, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250287, "file_size": 37252402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3856846899, "table_properties": {"data_size": 31020432, "index_size": 1779008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26010236, "raw_average_key_size": 28, "raw_value_size": 108207546, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928937, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317668, "oldest_key_time": 0, "file_creation_time": 1732317668, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:11.591214 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst) /dev/shm/ToplingTemp/Topling-445932-A3bqXs 2024/11/23-07:21:12.511713 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst): first pass time = 0.92's, 145.816'MB/sec 2024/11/23-07:21:13.825493 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst): old prealloc_size = 147639500, real_size = 32677648 2024/11/23-07:21:13.826065 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928750 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.51 avg-zval = 33.03 2024/11/23-07:21:13.827716 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:14.747196 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250288.sst): second pass time = 1.31's, 82.408'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.907'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.991'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 285.516'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34950.864'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.408'MB/sec zip pipeline throughput = 82.408'MB/sec entries = 928750 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.51 avg-zval = 33.03 usrkeys = 928750 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928750 bytes = 3250640 } seq expand size = 6501250 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5127 value = 3.5273 dict = 1.00 all = 3.6490 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 1.592862 GB avg = 0.121 KB (by entry num) total key len = 0.368625 GB avg = 0.028 KB total ukey len = 0.263303 GB avg = 0.020 KB total ukey num = 0.013165168 Billion total entry num = 0.013165168 Billion write speed all = 0.024236330 MB/sec (with seq num) write speed all = 0.022934969 MB/sec (without seq num) 2024/11/23-07:21:14.783961 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250288: 928750 keys, 36781676 bytes, temperature: kWarm 2024/11/23-07:21:14.784023 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317674783990, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250288, "file_size": 36781676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3856846260, "table_properties": {"data_size": 30678336, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26005000, "raw_average_key_size": 28, "raw_value_size": 108212812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317671, "oldest_key_time": 0, "file_creation_time": 1732317671, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:14.824522 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst) /dev/shm/ToplingTemp/Topling-445932-UAzFyv 2024/11/23-07:21:15.710028 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst): first pass time = 0.89's, 151.579'MB/sec 2024/11/23-07:21:17.021731 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst): old prealloc_size = 147639500, real_size = 31449328 2024/11/23-07:21:17.022327 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928837 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.50 avg-zval = 31.71 2024/11/23-07:21:17.024515 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-07:21:17.893924 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250289.sst): second pass time = 1.31's, 82.538'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.897'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.826'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 275.054'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31178.741'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.538'MB/sec zip pipeline throughput = 82.538'MB/sec entries = 928837 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 31.71 usrkeys = 928837 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 31.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928837 bytes = 3367056 } seq expand size = 6501859 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5122 value = 3.6744 dict = 1.00 all = 3.7628 } Zip/UnZip{ index = 0.0951 value = 0.2722 dict = 1.00 all = 0.2658 } ---------------------------- total value len = 1.707574 GB avg = 0.121 KB (by entry num) total key len = 0.394632 GB avg = 0.028 KB total ukey len = 0.281880 GB avg = 0.020 KB total ukey num = 0.014094005 Billion total entry num = 0.014094005 Billion write speed all = 0.025974067 MB/sec (with seq num) write speed all = 0.024580945 MB/sec (without seq num) 2024/11/23-07:21:17.940797 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250289: 928837 keys, 35669440 bytes, temperature: kWarm 2024/11/23-07:21:17.940860 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317677940829, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250289, "file_size": 35669440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3856846854, "table_properties": {"data_size": 29449760, "index_size": 1767168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007436, "raw_average_key_size": 28, "raw_value_size": 108210302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928837, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317674, "oldest_key_time": 0, "file_creation_time": 1732317674, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:17.975913 140024760219200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst) /dev/shm/ToplingTemp/Topling-445932-oGPllw 2024/11/23-07:21:18.172582 140024760219200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst): first pass time = 0.20's, 243.024'MB/sec 2024/11/23-07:21:18.639774 140024760219200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst): old prealloc_size = 147639500, real_size = 13246624 2024/11/23-07:21:18.640017 140024760219200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 331047 prefix = 4 raw-key = 0.0053 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0385 GB zip-val = 0.0125 GB avg-val = 116.35 avg-zval = 37.86 2024/11/23-07:21:18.640340 140024760219200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.048 GB 2024/11/23-07:21:18.953178 140024760219200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00/250290.sst): second pass time = 0.47's, 82.545'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.31's, 47.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2786.025'MB/sec (index lex order gen) rebuild zvType time = 0.27's, 0.310'MB/sec write SST data time = 0.05's, 320.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.395'MB, throughput = 13778.282'MB/sec zip my value time = 0.47's, unzip length = 0.039'GB zip my value throughput = 82.545'MB/sec zip pipeline throughput = 82.545'MB/sec entries = 331047 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.35 avg-zval = 37.86 usrkeys = 331047 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.35 avg-zval = 37.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 331047 bytes = 1200064 } seq expand size = 2317329 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0066 GB value = 0.0385 GB dict = 0.39 MB all = 0.0478 GB } __ZipSize{ index = 0.0006 GB value = 0.0125 GB dict = 0.39 MB all = 0.0148 GB } UnZip/Zip{ index = 10.4942 value = 3.0733 dict = 1.00 all = 3.2371 } Zip/UnZip{ index = 0.0953 value = 0.3254 dict = 1.00 all = 0.3089 } ---------------------------- total value len = 1.748407 GB avg = 0.121 KB (by entry num) total key len = 0.403901 GB avg = 0.028 KB total ukey len = 0.288501 GB avg = 0.020 KB total ukey num = 0.014425052 Billion total entry num = 0.014425052 Billion write speed all = 0.026592774 MB/sec (with seq num) write speed all = 0.025166948 MB/sec (without seq num) 2024/11/23-07:21:18.972405 140024760219200 [compaction_job.cc:1948] [default] [JOB 21761] Generated table #250290: 331047 keys, 14761781 bytes, temperature: kWarm 2024/11/23-07:21:18.972466 140024760219200 EVENT_LOG_v1 {"time_micros": 1732317678972434, "cf_name": "default", "job": 21761, "event": "table_file_creation", "file_number": 250290, "file_size": 14761781, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3856846292, "table_properties": {"data_size": 12532752, "index_size": 630912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9269316, "raw_average_key_size": 28, "raw_value_size": 38516470, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 331047, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732317677, "oldest_key_time": 0, "file_creation_time": 1732317677, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 250290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-07:21:18.987853 140024760219200 [compaction_job.cc:703] job-21761: subcompact[0], size: 0.568871 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21761/att-00 [250275,250276,250277,250278,250279,250280,250281,250282,250283,250284,250285,250286,250287,250288,250289,250290] 2024/11/23-07:21:19.085517 140024760219200 (Original Log Time 2024/11/23-07:20:30.092262) EVENT_LOG_v1 {"time_micros": 1732317630092221, "job": 21761, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [250207], "files_L3": [250259, 250260, 250151, 250168, 250169, 250170, 250171, 250172, 250173, 250174, 250175, 250176, 250177, 250178, 250179, 250261], "score": 0.98839, "input_data_size": 595651775, "oldest_snapshot_seqno": 3857171310} 2024/11/23-07:21:19.085521 140024760219200 (Original Log Time 2024/11/23-07:21:19.084713) [compaction_job.cc:2022] [default] [JOB 21761] Compacted 1@2 + 16@3 files to L3 => 568871071 bytes 2024/11/23-07:21:19.085523 140024760219200 (Original Log Time 2024/11/23-07:21:19.085445) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 12.2 rd, 11.6 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 540.9 +0.0 blob) out(542.5 +0.0 blob), read-write-amplify(40.9) write-amplify(20.0) OK, records in: 14493609, records dropped: 68557 output_compression: Snappy 2024/11/23-07:21:19.085528 140024760219200 (Original Log Time 2024/11/23-07:21:19.085497) EVENT_LOG_v1 {"time_micros": 1732317679085469, "cf": "default", "job": 21761, "event": "compaction_finished", "compaction_time_micros": 48895598, "compaction_time_cpu_micros": 22250525, "output_level": 3, "num_output_files": 16, "total_output_size": 568871071, "num_input_records": 14493609, "num_output_records": 14425052, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 354801186, "file_range_sync_nanos": 0, "file_fsync_nanos": 14952, "file_prepare_write_nanos": 34907, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-07:21:19.200203 140024760219200 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732317629,"executesMs":49139,"compactionJobId":21761,"attempt":0,"compactionInputRawBytes":2070787499,"compactionInputZipBytes":595651775,"compactionOutputRawBytes":0,"compactionOutputZipBytes":568871071,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-cd4696ba15757bd848ef8c4bb44b4f15-27646fbaa51cc9e2-00"}}, response = 2024/11/23-07:21:19.200274 140024760219200 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21761/att-00: olev 3, work 49.139 s, result 6.348 ms, install 0.834 ms, input{raw 1.929 GiB zip 568.058 MiB}