2024/11/23-15:58:27.287786 139957693548096 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00: fork to child time = 0.025091 sec 2024/11/23-15:58:27.288865 139957693548096 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00 2024/11/23-15:58:27.298258 139957693548096 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00 2024/11/23-15:58:27.299460 139957693548096 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:58:27.299811 139957693548096 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:58:27.354316 139957693548096 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29986/att-00/MANIFEST-206347 2024/11/23-15:58:27.387024 139957693548096 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29986/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 268060, last_sequence is 4089533533, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 267920 2024/11/23-15:58:27.387034 139957693548096 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:58:27.387215 139957693548096 [version_set.cc:5721] Creating manifest 268060 2024/11/23-15:58:27.422408 139957693548096 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:58:27.446635 139957693548096 [compaction_job.cc:2437] [default] [JOB 29986] Compacting 1@2 + 21@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-15:58:27.446667 139957693548096 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 22422 Base level 2, inputs: [268048(31MB)], [267980(40MB) 267981(40MB) 267982(40MB) 267983(40MB) 267984(40MB) 267985(40MB) 267986(40MB) 267987(40MB) 267988(40MB) 267989(40MB) 267990(40MB) 267991(40MB) 267992(40MB) 267993(40MB) 267994(40MB) 267995(40MB) 267996(40MB) 267997(40MB) 267998(40MB) 267999(39MB) 268000(40MB)] 2024/11/23-15:58:27.456277 139957693548096 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:58:27.456319 139957693548096 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:58:27.456348 139957693548096 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:58:27.456357 139957693548096 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:58:27.456371 139957693548096 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:58:28.169172 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst) /dev/shm/ToplingTemp/Topling-104554-KkrR35 2024/11/23-15:58:28.392064 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst): first pass time = 0.22's, 602.326'MB/sec 2024/11/23-15:58:30.747485 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst): old prealloc_size = 147639500, real_size = 40646992 2024/11/23-15:58:30.747921 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 311.99 2024/11/23-15:58:30.748142 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:30.941543 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268061.sst): second pass time = 2.35's, 55.919'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 218.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 563.642'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.231'MB/sec write SST data time = 0.05's, 790.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38815.795'MB/sec zip my value time = 2.35's, unzip length = 0.132'GB zip my value throughput = 55.919'MB/sec zip pipeline throughput = 110.917'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 311.99 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 311.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6470 value = 3.2597 dict = 1.00 all = 3.1596 } Zip/UnZip{ index = 0.1504 value = 0.3068 dict = 1.00 all = 0.3165 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.410436523 MB/sec (with seq num) write speed all = 0.393904168 MB/sec (without seq num) 2024/11/23-15:58:30.993253 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268061: 129431 keys, 42479875 bytes, temperature: kWarm 2024/11/23-15:58:30.993364 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348710993292, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268061, "file_size": 42479875, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4089484516, "table_properties": {"data_size": 40380800, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629634, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348708, "oldest_key_time": 0, "file_creation_time": 1732348708, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268061, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:31.050306 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst) /dev/shm/ToplingTemp/Topling-104554-5Hrjn3 2024/11/23-15:58:31.722428 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst): first pass time = 0.67's, 199.703'MB/sec 2024/11/23-15:58:32.690294 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst): old prealloc_size = 147639500, real_size = 40841344 2024/11/23-15:58:32.690445 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.98 avg-zval = 313.54 2024/11/23-15:58:32.690639 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:32.909419 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268062.sst): second pass time = 0.97's, 136.113'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 194.768'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1761.427'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.191'MB/sec write SST data time = 0.05's, 859.955'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44396.193'MB/sec zip my value time = 0.97's, unzip length = 0.132'GB zip my value throughput = 136.113'MB/sec zip pipeline throughput = 165.592'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1016.98 avg-zval = 313.54 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1023.98 avg-zval = 313.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8555 value = 3.2435 dict = 1.00 all = 3.1451 } Zip/UnZip{ index = 0.1459 value = 0.3083 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.415724412 MB/sec (with seq num) write speed all = 0.399152574 MB/sec (without seq num) 2024/11/23-15:58:32.962436 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268062: 129431 keys, 42674540 bytes, temperature: kWarm 2024/11/23-15:58:32.962505 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348712962470, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268062, "file_size": 42674540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4089474945, "table_properties": {"data_size": 40582256, "index_size": 226560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629120, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348711, "oldest_key_time": 0, "file_creation_time": 1732348711, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268062, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:33.006686 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst) /dev/shm/ToplingTemp/Topling-104554-hLQkR5 2024/11/23-15:58:34.146747 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst): first pass time = 1.14's, 117.733'MB/sec 2024/11/23-15:58:35.426061 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst): old prealloc_size = 147639500, real_size = 40596672 2024/11/23-15:58:35.426197 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.59 2024/11/23-15:58:35.426353 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:35.593521 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268063.sst): second pass time = 1.28's, 103.061'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2030.858'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.264'MB/sec write SST data time = 0.04's, 944.059'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36310.636'MB/sec zip my value time = 1.28's, unzip length = 0.132'GB zip my value throughput = 103.061'MB/sec zip pipeline throughput = 124.838'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.98 avg-zval = 311.59 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.98 avg-zval = 311.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6217 value = 3.2638 dict = 1.00 all = 3.1633 } Zip/UnZip{ index = 0.1510 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938729 Billion total entry num = 0.052746458 Billion write speed all = 0.420999631 MB/sec (with seq num) write speed all = 0.404388784 MB/sec (without seq num) 2024/11/23-15:58:35.647755 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268063: 129432 keys, 42429371 bytes, temperature: kWarm 2024/11/23-15:58:35.647812 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348715647779, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268063, "file_size": 42429371, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873554, "largest_seqno": 4089479208, "table_properties": {"data_size": 40329584, "index_size": 234560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629578, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348712, "oldest_key_time": 0, "file_creation_time": 1732348712, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268063, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:35.685843 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst) /dev/shm/ToplingTemp/Topling-104554-CzImK5 2024/11/23-15:58:36.223743 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst): first pass time = 0.54's, 249.536'MB/sec 2024/11/23-15:58:37.044155 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst): old prealloc_size = 147639500, real_size = 40862768 2024/11/23-15:58:37.044278 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.95 avg-zval = 313.71 2024/11/23-15:58:37.044454 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:37.211860 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268064.sst): second pass time = 0.82's, 160.622'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2288.032'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 889.231'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46228.849'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.622'MB/sec zip pipeline throughput = 160.622'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.95 avg-zval = 313.71 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.95 avg-zval = 313.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8829 value = 3.2417 dict = 1.00 all = 3.1436 } Zip/UnZip{ index = 0.1453 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.378761 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068164 Billion total entry num = 0.052875893 Billion write speed all = 0.426291606 MB/sec (with seq num) write speed all = 0.409641057 MB/sec (without seq num) 2024/11/23-15:58:37.264724 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268064: 129435 keys, 42695649 bytes, temperature: kWarm 2024/11/23-15:58:37.264797 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348717264758, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268064, "file_size": 42695649, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4089492591, "table_properties": {"data_size": 40604576, "index_size": 225664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629156, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348715, "oldest_key_time": 0, "file_creation_time": 1732348715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268064, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:37.303186 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst) /dev/shm/ToplingTemp/Topling-104554-2Fnbn6 2024/11/23-15:58:38.112578 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst): first pass time = 0.81's, 165.834'MB/sec 2024/11/23-15:58:39.409585 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst): old prealloc_size = 147639500, real_size = 40598304 2024/11/23-15:58:39.409715 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.61 2024/11/23-15:58:39.409889 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:39.573338 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268065.sst): second pass time = 1.30's, 101.538'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2199.105'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 975.838'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29095.385'MB/sec zip my value time = 1.30's, unzip length = 0.132'GB zip my value throughput = 101.538'MB/sec zip pipeline throughput = 134.130'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1017.00 avg-zval = 311.61 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1024.00 avg-zval = 311.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6143 value = 3.2637 dict = 1.00 all = 3.1632 } Zip/UnZip{ index = 0.1512 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197593 Billion total entry num = 0.053005322 Billion write speed all = 0.431570299 MB/sec (with seq num) write speed all = 0.414880544 MB/sec (without seq num) 2024/11/23-15:58:39.625082 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268065: 129429 keys, 42431187 bytes, temperature: kWarm 2024/11/23-15:58:39.625147 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348719625114, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268065, "file_size": 42431187, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4089490572, "table_properties": {"data_size": 40330960, "index_size": 234816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629757, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348717, "oldest_key_time": 0, "file_creation_time": 1732348717, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268065, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:39.667298 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst) /dev/shm/ToplingTemp/Topling-104554-Njv5h3 2024/11/23-15:58:40.336798 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst): first pass time = 0.67's, 200.486'MB/sec 2024/11/23-15:58:41.110108 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst): old prealloc_size = 147639500, real_size = 40715264 2024/11/23-15:58:41.110233 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.99 avg-zval = 312.52 2024/11/23-15:58:41.110391 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:41.273266 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268066.sst): second pass time = 0.77's, 170.367'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.799'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2230.703'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 959.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42008.109'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.367'MB/sec zip pipeline throughput = 170.367'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 312.52 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 312.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6762 value = 3.2541 dict = 1.00 all = 3.1545 } Zip/UnZip{ index = 0.1498 value = 0.3073 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 9.643833 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327023 Billion total entry num = 0.053134752 Billion write speed all = 0.436859376 MB/sec (with seq num) write speed all = 0.420129987 MB/sec (without seq num) 2024/11/23-15:58:41.324314 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268066: 129430 keys, 42547997 bytes, temperature: kWarm 2024/11/23-15:58:41.324390 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348721324349, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268066, "file_size": 42547997, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237881756, "largest_seqno": 4089490300, "table_properties": {"data_size": 40450096, "index_size": 232640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629499, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348719, "oldest_key_time": 0, "file_creation_time": 1732348719, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268066, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:41.365992 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst) /dev/shm/ToplingTemp/Topling-104554-cEyAi4 2024/11/23-15:58:41.931456 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst): first pass time = 0.57's, 237.378'MB/sec 2024/11/23-15:58:42.828820 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst): old prealloc_size = 147639500, real_size = 41033936 2024/11/23-15:58:42.828940 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.01 avg-zval = 315.06 2024/11/23-15:58:42.829125 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:42.992279 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268067.sst): second pass time = 0.90's, 146.903'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.280'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2269.858'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 952.535'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 52129.971'MB/sec zip my value time = 0.90's, unzip length = 0.132'GB zip my value throughput = 146.903'MB/sec zip pipeline throughput = 146.903'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 315.06 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.01 avg-zval = 315.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9396 value = 3.2280 dict = 1.00 all = 3.1311 } Zip/UnZip{ index = 0.1441 value = 0.3098 dict = 1.00 all = 0.3194 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456451 Billion total entry num = 0.053264180 Billion write speed all = 0.442147432 MB/sec (with seq num) write speed all = 0.425378428 MB/sec (without seq num) 2024/11/23-15:58:43.043821 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268067: 129428 keys, 42866446 bytes, temperature: kWarm 2024/11/23-15:58:43.043890 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348723043857, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268067, "file_size": 42866446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237880987, "largest_seqno": 4089490021, "table_properties": {"data_size": 40777600, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630038, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348721, "oldest_key_time": 0, "file_creation_time": 1732348721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268067, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:43.084766 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst) /dev/shm/ToplingTemp/Topling-104554-PYYR14 2024/11/23-15:58:43.995534 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst): first pass time = 0.91's, 147.376'MB/sec 2024/11/23-15:58:45.819855 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst): old prealloc_size = 147639500, real_size = 40982832 2024/11/23-15:58:45.820055 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.99 avg-zval = 314.68 2024/11/23-15:58:45.820245 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:45.987282 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268068.sst): second pass time = 1.82's, 72.182'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.750'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1307.762'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 926.974'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44051.750'MB/sec zip my value time = 1.82's, unzip length = 0.132'GB zip my value throughput = 72.182'MB/sec zip pipeline throughput = 133.597'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 314.68 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.99 avg-zval = 314.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 7.0322 value = 3.2318 dict = 1.00 all = 3.1348 } Zip/UnZip{ index = 0.1422 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 9.908904 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585881 Billion total entry num = 0.053393610 Billion write speed all = 0.447412285 MB/sec (with seq num) write speed all = 0.430604514 MB/sec (without seq num) 2024/11/23-15:58:46.097501 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268068: 129430 keys, 42815790 bytes, temperature: kWarm 2024/11/23-15:58:46.097576 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348726097527, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268068, "file_size": 42815790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4089493130, "table_properties": {"data_size": 40729440, "index_size": 220864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629396, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348723, "oldest_key_time": 0, "file_creation_time": 1732348723, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268068, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:46.147148 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst) /dev/shm/ToplingTemp/Topling-104554-EyCNd7 2024/11/23-15:58:46.789834 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst): first pass time = 0.64's, 208.854'MB/sec 2024/11/23-15:58:47.609643 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst): old prealloc_size = 147639500, real_size = 40780224 2024/11/23-15:58:47.609772 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.97 avg-zval = 313.04 2024/11/23-15:58:47.609930 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:47.774094 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268069.sst): second pass time = 0.82's, 160.704'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2260.305'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 929.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43392.492'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.704'MB/sec zip pipeline throughput = 160.704'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1016.97 avg-zval = 313.04 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1023.97 avg-zval = 313.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7601 value = 3.2487 dict = 1.00 all = 3.1497 } Zip/UnZip{ index = 0.1479 value = 0.3078 dict = 1.00 all = 0.3175 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715314 Billion total entry num = 0.053523043 Billion write speed all = 0.452697420 MB/sec (with seq num) write speed all = 0.435850089 MB/sec (without seq num) 2024/11/23-15:58:47.829942 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268069: 129433 keys, 42612762 bytes, temperature: kWarm 2024/11/23-15:58:47.830027 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348727829989, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268069, "file_size": 42612762, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4089493136, "table_properties": {"data_size": 40517936, "index_size": 229760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629413, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348726, "oldest_key_time": 0, "file_creation_time": 1732348726, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268069, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:47.872712 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst) /dev/shm/ToplingTemp/Topling-104554-PTYnf6 2024/11/23-15:58:48.521503 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst): first pass time = 0.65's, 206.886'MB/sec 2024/11/23-15:58:49.396887 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst): old prealloc_size = 147639500, real_size = 40593312 2024/11/23-15:58:49.397069 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.57 2024/11/23-15:58:49.397268 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:49.621787 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268070.sst): second pass time = 0.87's, 150.510'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 188.654'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1472.247'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.185'MB/sec write SST data time = 0.05's, 855.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49713.286'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 150.510'MB/sec zip pipeline throughput = 159.639'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 311.57 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6144 value = 3.2641 dict = 1.00 all = 3.1636 } Zip/UnZip{ index = 0.1512 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173975 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844744 Billion total entry num = 0.053652473 Billion write speed all = 0.457980715 MB/sec (with seq num) write speed all = 0.441093871 MB/sec (without seq num) 2024/11/23-15:58:49.674614 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268070: 129430 keys, 42425817 bytes, temperature: kWarm 2024/11/23-15:58:49.674680 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348729674648, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268070, "file_size": 42425817, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873605, "largest_seqno": 4089487703, "table_properties": {"data_size": 40325968, "index_size": 234816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629492, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348727, "oldest_key_time": 0, "file_creation_time": 1732348727, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268070, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:49.718557 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst) /dev/shm/ToplingTemp/Topling-104554-qdgyi7 2024/11/23-15:58:50.226933 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst): first pass time = 0.51's, 264.031'MB/sec 2024/11/23-15:58:51.034251 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst): old prealloc_size = 147639500, real_size = 40684512 2024/11/23-15:58:51.034375 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.28 2024/11/23-15:58:51.034530 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:51.200327 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268071.sst): second pass time = 0.81's, 163.335'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.042'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2305.822'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 909.598'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 45612.758'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.335'MB/sec zip pipeline throughput = 163.335'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 312.28 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 312.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6579 value = 3.2566 dict = 1.00 all = 3.1567 } Zip/UnZip{ index = 0.1502 value = 0.3071 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 10.306510 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974174 Billion total entry num = 0.053781903 Billion write speed all = 0.463268141 MB/sec (with seq num) write speed all = 0.446341611 MB/sec (without seq num) 2024/11/23-15:58:51.252943 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268071: 129430 keys, 42517863 bytes, temperature: kWarm 2024/11/23-15:58:51.253029 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348731252990, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268071, "file_size": 42517863, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4089489489, "table_properties": {"data_size": 40418704, "index_size": 233280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629327, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348729, "oldest_key_time": 0, "file_creation_time": 1732348729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268071, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:51.293622 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst) /dev/shm/ToplingTemp/Topling-104554-p2afL3 2024/11/23-15:58:51.937118 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst): first pass time = 0.64's, 208.588'MB/sec 2024/11/23-15:58:52.832907 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst): old prealloc_size = 147639500, real_size = 41025360 2024/11/23-15:58:52.833065 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 314.99 2024/11/23-15:58:52.833241 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:53.090749 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268072.sst): second pass time = 0.90's, 147.069'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.26's, 166.251'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1646.737'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.173'MB/sec write SST data time = 0.07's, 604.150'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34394.018'MB/sec zip my value time = 0.90's, unzip length = 0.132'GB zip my value throughput = 147.069'MB/sec zip pipeline throughput = 147.069'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.00 avg-zval = 314.99 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.00 avg-zval = 314.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9258 value = 3.2287 dict = 1.00 all = 3.1317 } Zip/UnZip{ index = 0.1444 value = 0.3097 dict = 1.00 all = 0.3193 } ---------------------------- total value len = 10.439046 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103603 Billion total entry num = 0.053911332 Billion write speed all = 0.468549176 MB/sec (with seq num) write speed all = 0.451583174 MB/sec (without seq num) 2024/11/23-15:58:53.150162 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268072: 129429 keys, 42857745 bytes, temperature: kWarm 2024/11/23-15:58:53.150253 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348733150203, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268072, "file_size": 42857745, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4089467988, "table_properties": {"data_size": 40768576, "index_size": 224256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629727, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348731, "oldest_key_time": 0, "file_creation_time": 1732348731, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268072, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:53.203202 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst) /dev/shm/ToplingTemp/Topling-104554-F31Nf7 2024/11/23-15:58:53.909248 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst): first pass time = 0.71's, 190.110'MB/sec 2024/11/23-15:58:54.730229 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst): old prealloc_size = 147639500, real_size = 40587840 2024/11/23-15:58:54.730406 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.55 2024/11/23-15:58:54.730638 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:54.902265 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268073.sst): second pass time = 0.82's, 160.483'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1586.390'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.263'MB/sec write SST data time = 0.05's, 870.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44895.094'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.483'MB/sec zip pipeline throughput = 160.483'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1017.02 avg-zval = 311.55 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1022.92 avg-zval = 311.55 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 109048 bytes = 408944 } seq expand size = 763336 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6890 value = 3.2644 dict = 1.00 all = 3.1682 } Zip/UnZip{ index = 0.1495 value = 0.3063 dict = 1.00 all = 0.3156 } ---------------------------- total value len = 10.571439 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233030 Billion total entry num = 0.054040759 Billion write speed all = 0.473825274 MB/sec (with seq num) write speed all = 0.456819752 MB/sec (without seq num) 2024/11/23-15:58:54.955112 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268073: 129427 keys, 42364505 bytes, temperature: kWarm 2024/11/23-15:58:54.955181 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348734955145, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268073, "file_size": 42364505, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089490485, "table_properties": {"data_size": 40323120, "index_size": 232192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629500, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348733, "oldest_key_time": 0, "file_creation_time": 1732348733, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268073, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:54.998987 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst) /dev/shm/ToplingTemp/Topling-104554-J6sYg3 2024/11/23-15:58:55.543430 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst): first pass time = 0.54's, 246.540'MB/sec 2024/11/23-15:58:56.353524 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst): old prealloc_size = 147639500, real_size = 40646400 2024/11/23-15:58:56.353662 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.98 avg-zval = 312.08 2024/11/23-15:58:56.353779 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:56.518103 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268074.sst): second pass time = 0.81's, 162.646'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 256.422'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2118.977'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.05's, 877.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46596.003'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.646'MB/sec zip pipeline throughput = 162.646'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.98 avg-zval = 312.08 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1019.52 avg-zval = 312.08 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 46992 bytes = 176240 } seq expand size = 328944 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 7.0466 value = 3.2587 dict = 1.00 all = 3.1813 } Zip/UnZip{ index = 0.1419 value = 0.3069 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 10.703397 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362462 Billion total entry num = 0.054170191 Billion write speed all = 0.479087229 MB/sec (with seq num) write speed all = 0.462042061 MB/sec (without seq num) 2024/11/23-15:58:56.575388 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268074: 129432 keys, 42190288 bytes, temperature: kWarm 2024/11/23-15:58:56.575489 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348736575440, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268074, "file_size": 42190288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089399765, "table_properties": {"data_size": 40393456, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629586, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348734, "oldest_key_time": 0, "file_creation_time": 1732348734, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268074, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:56.635860 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst) /dev/shm/ToplingTemp/Topling-104554-lmS9K4 2024/11/23-15:58:57.225444 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst): first pass time = 0.59's, 227.670'MB/sec 2024/11/23-15:58:58.045121 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst): old prealloc_size = 147639500, real_size = 40924128 2024/11/23-15:58:58.045243 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.21 2024/11/23-15:58:58.045387 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:58.210029 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268075.sst): second pass time = 0.82's, 160.892'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.894'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2282.492'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 940.981'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40714.113'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.892'MB/sec zip pipeline throughput = 160.892'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 314.21 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1022.68 avg-zval = 314.21 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 104796 bytes = 392992 } seq expand size = 733572 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9515 value = 3.2367 dict = 1.00 all = 3.1444 } Zip/UnZip{ index = 0.1439 value = 0.3090 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 10.835760 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491890 Billion total entry num = 0.054299619 Billion write speed all = 0.484362968 MB/sec (with seq num) write speed all = 0.467278211 MB/sec (without seq num) 2024/11/23-15:58:58.262905 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268075: 129428 keys, 42684947 bytes, temperature: kWarm 2024/11/23-15:58:58.262971 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348738262937, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268075, "file_size": 42684947, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089476364, "table_properties": {"data_size": 40668176, "index_size": 223424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629365, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348736, "oldest_key_time": 0, "file_creation_time": 1732348736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268075, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:58:58.302294 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst) /dev/shm/ToplingTemp/Topling-104554-xxr9g5 2024/11/23-15:58:58.900521 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst): first pass time = 0.60's, 224.380'MB/sec 2024/11/23-15:58:59.765331 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst): old prealloc_size = 147639500, real_size = 40960368 2024/11/23-15:58:59.765455 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.51 2024/11/23-15:58:59.765613 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:58:59.932044 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268076.sst): second pass time = 0.86's, 152.471'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2209.248'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 905.253'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42455.145'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.471'MB/sec zip pipeline throughput = 152.471'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1017.02 avg-zval = 314.51 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1022.54 avg-zval = 314.51 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 102115 bytes = 382944 } seq expand size = 714805 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9996 value = 3.2337 dict = 1.00 all = 3.1425 } Zip/UnZip{ index = 0.1429 value = 0.3092 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 10.968105 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621317 Billion total entry num = 0.054429046 Billion write speed all = 0.489636694 MB/sec (with seq num) write speed all = 0.472512374 MB/sec (without seq num) 2024/11/23-15:58:59.983957 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268076: 129427 keys, 42711149 bytes, temperature: kWarm 2024/11/23-15:58:59.984037 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348739984001, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268076, "file_size": 42711149, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089489297, "table_properties": {"data_size": 40705952, "index_size": 221888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629529, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348738, "oldest_key_time": 0, "file_creation_time": 1732348738, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:00.023124 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst) /dev/shm/ToplingTemp/Topling-104554-4hTVu6 2024/11/23-15:59:00.771096 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst): first pass time = 0.75's, 179.457'MB/sec 2024/11/23-15:59:01.570535 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst): old prealloc_size = 147639500, real_size = 40691568 2024/11/23-15:59:01.570678 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 312.37 2024/11/23-15:59:01.570820 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:59:01.734249 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268077.sst): second pass time = 0.80's, 164.809'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2225.600'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 924.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44920.963'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.809'MB/sec zip pipeline throughput = 164.809'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.01 avg-zval = 312.37 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1022.18 avg-zval = 312.37 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 95655 bytes = 358720 } seq expand size = 669585 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.7825 value = 3.2558 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1474 value = 0.3071 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 11.100404 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750746 Billion total entry num = 0.054558475 Billion write speed all = 0.494906391 MB/sec (with seq num) write speed all = 0.477742567 MB/sec (without seq num) 2024/11/23-15:59:01.786508 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268077: 129429 keys, 42418392 bytes, temperature: kWarm 2024/11/23-15:59:01.786589 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348741786552, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268077, "file_size": 42418392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089492453, "table_properties": {"data_size": 40430048, "index_size": 228992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630136, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348740, "oldest_key_time": 0, "file_creation_time": 1732348740, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:01.826744 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst) /dev/shm/ToplingTemp/Topling-104554-f4RBt5 2024/11/23-15:59:02.446177 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst): first pass time = 0.62's, 216.697'MB/sec 2024/11/23-15:59:03.292741 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst): old prealloc_size = 147639500, real_size = 40458608 2024/11/23-15:59:03.292866 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.63 2024/11/23-15:59:03.292948 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:59:03.449955 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268078.sst): second pass time = 0.85's, 155.751'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2163.173'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.281'MB/sec write SST data time = 0.04's, 1001.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 44879.333'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.751'MB/sec zip pipeline throughput = 155.751'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.63 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.56 avg-zval = 310.63 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 28746 bytes = 107808 } seq expand size = 201222 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9936 value = 3.2740 dict = 1.00 all = 3.2006 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3124 } ---------------------------- total value len = 11.232235 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880175 Billion total entry num = 0.054687904 Billion write speed all = 0.500158612 MB/sec (with seq num) write speed all = 0.482955231 MB/sec (without seq num) 2024/11/23-15:59:03.500470 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268078: 129429 keys, 41934897 bytes, temperature: kWarm 2024/11/23-15:59:03.500549 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348743500506, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268078, "file_size": 41934897, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089478684, "table_properties": {"data_size": 40204000, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629789, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348741, "oldest_key_time": 0, "file_creation_time": 1732348741, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268078, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:03.545154 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst) /dev/shm/ToplingTemp/Topling-104554-rWv566 2024/11/23-15:59:04.129885 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst): first pass time = 0.58's, 229.556'MB/sec 2024/11/23-15:59:04.951967 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst): old prealloc_size = 147639500, real_size = 40591408 2024/11/23-15:59:04.952106 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.66 2024/11/23-15:59:04.952187 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:59:05.111880 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268079.sst): second pass time = 0.82's, 160.261'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.107'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2354.663'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 1004.950'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 31829.672'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.261'MB/sec zip pipeline throughput = 160.261'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.66 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.56 avg-zval = 311.66 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 28441 bytes = 106672 } seq expand size = 199087 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2633 dict = 1.00 all = 3.1907 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3134 } ---------------------------- total value len = 11.364064 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009602 Billion total entry num = 0.054817331 Billion write speed all = 0.505411109 MB/sec (with seq num) write speed all = 0.488168140 MB/sec (without seq num) 2024/11/23-15:59:05.162879 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268079: 129427 keys, 42065809 bytes, temperature: kWarm 2024/11/23-15:59:05.162946 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348745162912, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268079, "file_size": 42065809, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089432448, "table_properties": {"data_size": 40336800, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629799, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348743, "oldest_key_time": 0, "file_creation_time": 1732348743, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268079, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:05.201417 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst) /dev/shm/ToplingTemp/Topling-104554-2qRs73 2024/11/23-15:59:05.846799 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst): first pass time = 0.65's, 207.980'MB/sec 2024/11/23-15:59:06.644265 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst): old prealloc_size = 147639500, real_size = 40456608 2024/11/23-15:59:06.644394 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.61 2024/11/23-15:59:06.644478 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:59:06.804698 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268080.sst): second pass time = 0.80's, 165.219'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.362'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2113.478'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 953.026'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 30506.720'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 165.219'MB/sec zip pipeline throughput = 165.219'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.61 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.49 avg-zval = 310.61 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 27786 bytes = 104208 } seq expand size = 194502 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9938 value = 3.2742 dict = 1.00 all = 3.2011 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3124 } ---------------------------- total value len = 11.495889 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139033 Billion total entry num = 0.054946762 Billion write speed all = 0.510662120 MB/sec (with seq num) write speed all = 0.493379589 MB/sec (without seq num) 2024/11/23-15:59:06.856100 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268080: 129431 keys, 41928360 bytes, temperature: kWarm 2024/11/23-15:59:06.856166 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348746856135, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268080, "file_size": 41928360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089486895, "table_properties": {"data_size": 40202000, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629779, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348745, "oldest_key_time": 0, "file_creation_time": 1732348745, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268080, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:06.893411 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst) /dev/shm/ToplingTemp/Topling-104554-PwqYf4 2024/11/23-15:59:07.428245 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst): first pass time = 0.53's, 250.968'MB/sec 2024/11/23-15:59:08.327518 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst): old prealloc_size = 147639500, real_size = 40839872 2024/11/23-15:59:08.327653 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.02 avg-zval = 313.58 2024/11/23-15:59:08.327786 139957693548096 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:59:08.488061 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268081.sst): second pass time = 0.90's, 146.496'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.170'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2231.354'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 985.336'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49821.494'MB/sec zip my value time = 0.90's, unzip length = 0.132'GB zip my value throughput = 146.496'MB/sec zip pipeline throughput = 146.496'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 313.58 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1022.18 avg-zval = 313.58 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 95435 bytes = 357904 } seq expand size = 668045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.9935 value = 3.2433 dict = 1.00 all = 3.1532 } Zip/UnZip{ index = 0.1430 value = 0.3083 dict = 1.00 all = 0.3171 } ---------------------------- total value len = 11.628186 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268460 Billion total entry num = 0.055076189 Billion write speed all = 0.515931220 MB/sec (with seq num) write speed all = 0.498609126 MB/sec (without seq num) 2024/11/23-15:59:08.539486 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268081: 129427 keys, 42565667 bytes, temperature: kWarm 2024/11/23-15:59:08.539562 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348748539520, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268081, "file_size": 42565667, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4089492806, "table_properties": {"data_size": 40585264, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629305, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732348746, "oldest_key_time": 0, "file_creation_time": 1732348746, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268081, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:08.577026 139957693548096 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268082.sst) /dev/shm/ToplingTemp/Topling-104554-y1vB94 2024/11/23-15:59:08.585469 139957693548096 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268082.sst): first pass time = 0.01's, 376.012'MB/sec 2024/11/23-15:59:08.615890 139957693548096 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268082.sst): old prealloc_size = 147639500, real_size = 1002800 2024/11/23-15:59:08.615932 139957693548096 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268082.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 3048 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.26 raw-val = 0.0031 GB zip-val = 0.0010 GB avg-val =1017.07 avg-zval = 328.42 2024/11/23-15:59:08.619568 139957693548096 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00/268082.sst): second pass time = 0.03's, 103.567'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 286.472'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 32.478'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.313'MB/sec write SST data time = 0.00's, 873.755'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.033'MB, throughput = 1564.835'MB/sec zip my value time = 0.03's, unzip length = 0.003'GB zip my value throughput = 103.567'MB/sec zip pipeline throughput = 103.567'MB/sec entries = 3048 avg-key = 20.00 avg-zkey = 0.27 avg-val = 1017.07 avg-zval = 328.42 usrkeys = 3048 avg-key = 12.00 avg-zkey = 0.27 avg-val = 1024.07 avg-zval = 328.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 3048 bytes = 11440 } seq expand size = 21336 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0000 GB value = 0.0031 GB dict = 0.03 MB all = 0.0032 GB } __ZipSize{ index = 0.0000 GB value = 0.0010 GB dict = 0.03 MB all = 0.0010 GB } UnZip/Zip{ index = 43.9615 value = 3.0968 dict = 1.00 all = 3.0189 } Zip/UnZip{ index = 0.0227 value = 0.3229 dict = 1.00 all = 0.3312 } ---------------------------- total value len = 11.631307 GB avg = 0.211 KB (by entry num) total key len = 1.495243 GB avg = 0.027 KB total ukey len = 1.030531 GB avg = 0.015 KB total ukey num = 0.068271508 Billion total entry num = 0.055079237 Billion write speed all = 0.516053661 MB/sec (with seq num) write speed all = 0.498730698 MB/sec (without seq num) 2024/11/23-15:59:08.628915 139957693548096 [compaction_job.cc:1948] [default] [JOB 29986] Generated table #268082: 3048 keys, 1047048 bytes, temperature: kWarm 2024/11/23-15:59:08.628971 139957693548096 EVENT_LOG_v1 {"time_micros": 1732348748628941, "cf_name": "default", "job": 29986, "event": "table_file_creation", "file_number": 268082, "file_size": 1047048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3356491908, "largest_seqno": 4087979167, "table_properties": {"data_size": 1001024, "index_size": 832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 60960, "raw_average_key_size": 20, "raw_value_size": 3100021, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 3048, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732348748, "oldest_key_time": 0, "file_creation_time": 1732348748, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 268082, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:59:08.641520 139957693548096 [compaction_job.cc:703] job-29986: subcompact[0], size: 0.893266 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29986/att-00 [268061,268062,268063,268064,268065,268066,268067,268068,268069,268070,268071,268072,268073,268074,268075,268076,268077,268078,268079,268080,268081,268082] 2024/11/23-15:59:08.775597 139957693548096 (Original Log Time 2024/11/23-15:58:27.446752) EVENT_LOG_v1 {"time_micros": 1732348707446689, "job": 29986, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [268048], "files_L3": [267980, 267981, 267982, 267983, 267984, 267985, 267986, 267987, 267988, 267989, 267990, 267991, 267992, 267993, 267994, 267995, 267996, 267997, 267998, 267999, 268000], "score": 0.989221, "input_data_size": 924942559, "oldest_snapshot_seqno": 4089533492} 2024/11/23-15:59:08.775992 139957693548096 (Original Log Time 2024/11/23-15:59:08.773919) [compaction_job.cc:2022] [default] [JOB 29986] Compacted 1@2 + 21@3 files to L3 => 893266104 bytes 2024/11/23-15:59:08.775995 139957693548096 (Original Log Time 2024/11/23-15:59:08.775510) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 22 0 0 0] max score 0.03, MB/sec: 22.5 rd, 21.7 wr, level 3, files in(1, 21) out(22 +0 blob) MB in(31.2, 850.9 +0.0 blob) out(851.9 +0.0 blob), read-write-amplify(55.6) write-amplify(27.3) OK, records in: 2749935, records dropped: 28862 output_compression: Snappy 2024/11/23-15:59:08.776002 139957693548096 (Original Log Time 2024/11/23-15:59:08.775568) EVENT_LOG_v1 {"time_micros": 1732348748775536, "cf": "default", "job": 29986, "event": "compaction_finished", "compaction_time_micros": 41194775, "compaction_time_cpu_micros": 8207127, "output_level": 3, "num_output_files": 22, "total_output_size": 893266104, "num_input_records": 2749935, "num_output_records": 2721073, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 630944015, "file_range_sync_nanos": 0, "file_fsync_nanos": 16002, "file_prepare_write_nanos": 48391, "lsm_state": [0, 0, 0, 22, 0, 0, 0]} 2024/11/23-15:59:08.907883 139957693548096 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732348707,"executesMs":41478,"compactionJobId":29986,"attempt":0,"compactionInputRawBytes":2851673602,"compactionInputZipBytes":924942559,"compactionOutputRawBytes":0,"compactionOutputZipBytes":893266104,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-09eb7e4315a4ee0e9a71d004847c28cb-60574b896576abd7-00"}}, response = 2024/11/23-15:59:08.907966 139957693548096 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29986/att-00: olev 3, work 41.478 s, result 7.735 ms, install 2.107 ms, input{raw 2.656 GiB zip 882.094 MiB}