2024/11/23-17:26:47.028965 139957257127488 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00: fork to child time = 0.007821 sec 2024/11/23-17:26:47.029827 139957257127488 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00 2024/11/23-17:26:47.039292 139957257127488 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00 2024/11/23-17:26:47.040204 139957257127488 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:26:47.040510 139957257127488 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:26:47.099102 139957257127488 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31903/att-00/MANIFEST-206347 2024/11/23-17:26:47.135010 139957257127488 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31903/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 272229, last_sequence is 4146795983, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 272086 2024/11/23-17:26:47.135021 139957257127488 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:26:47.135225 139957257127488 [version_set.cc:5721] Creating manifest 272229 2024/11/23-17:26:47.173709 139957257127488 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:26:47.195536 139957257127488 [compaction_job.cc:2437] [default] [JOB 31903] Compacting 1@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:26:47.195558 139957257127488 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23882 Base level 2, inputs: [272224(31MB)], [272143(40MB) 272144(40MB) 272145(40MB) 272146(40MB) 272147(40MB) 272148(40MB) 272149(40MB) 272150(40MB) 272151(40MB) 272152(40MB) 272153(40MB) 272154(40MB) 272155(40MB) 272156(40MB) 272157(40MB) 272158(40MB) 272159(40MB) 272160(40MB) 272161(40MB) 272162(40MB) 272163(40MB) 272164(40MB)] 2024/11/23-17:26:47.202583 139957257127488 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:26:47.202613 139957257127488 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:26:47.202632 139957257127488 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:26:47.202638 139957257127488 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:26:47.202649 139957257127488 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:26:47.894243 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst) /dev/shm/ToplingTemp/Topling-132551-LmJP13 2024/11/23-17:26:48.222624 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst): first pass time = 0.33's, 408.799'MB/sec 2024/11/23-17:26:49.148718 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst): old prealloc_size = 147639500, real_size = 40592576 2024/11/23-17:26:49.148853 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.57 2024/11/23-17:26:49.149276 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:26:49.329676 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272230.sst): second pass time = 0.93's, 142.257'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 234.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2143.470'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.248'MB/sec write SST data time = 0.05's, 839.622'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44419.028'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 142.257'MB/sec zip pipeline throughput = 142.257'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 311.57 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6470 value = 3.2641 dict = 1.00 all = 3.1636 } Zip/UnZip{ index = 0.1504 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.339594289 MB/sec (with seq num) write speed all = 0.325915455 MB/sec (without seq num) 2024/11/23-17:26:49.390636 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272230: 129431 keys, 42425104 bytes, temperature: kWarm 2024/11/23-17:26:49.390712 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354009390659, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272230, "file_size": 42425104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4146747996, "table_properties": {"data_size": 40326384, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629460, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354007, "oldest_key_time": 0, "file_creation_time": 1732354007, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272230, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:26:49.431508 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst) /dev/shm/ToplingTemp/Topling-132551-mWr4w6 2024/11/23-17:26:50.202324 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst): first pass time = 0.77's, 174.135'MB/sec 2024/11/23-17:26:51.840164 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst): old prealloc_size = 147639500, real_size = 40865968 2024/11/23-17:26:51.840318 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.74 2024/11/23-17:26:51.840520 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:26:52.264147 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272231.sst): second pass time = 1.64's, 80.482'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.42's, 100.711'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1764.649'MB/sec (index lex order gen) rebuild zvType time = 0.35's, 0.094'MB/sec write SST data time = 0.08's, 543.003'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43790.676'MB/sec zip my value time = 1.64's, unzip length = 0.132'GB zip my value throughput = 80.482'MB/sec zip pipeline throughput = 103.772'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.97 avg-zval = 313.74 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.97 avg-zval = 313.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9004 value = 3.2415 dict = 1.00 all = 3.1434 } Zip/UnZip{ index = 0.1449 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809299 Billion total entry num = 0.052617028 Billion write speed all = 0.343963279 MB/sec (with seq num) write speed all = 0.330252024 MB/sec (without seq num) 2024/11/23-17:26:52.318063 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272231: 129433 keys, 42698772 bytes, temperature: kWarm 2024/11/23-17:26:52.318133 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354012318096, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272231, "file_size": 42698772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4146760829, "table_properties": {"data_size": 40608352, "index_size": 225088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131630038, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354009, "oldest_key_time": 0, "file_creation_time": 1732354009, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272231, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:26:52.357643 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst) /dev/shm/ToplingTemp/Topling-132551-5yfr84 2024/11/23-17:26:53.032484 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst): first pass time = 0.67's, 198.899'MB/sec 2024/11/23-17:26:53.974073 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst): old prealloc_size = 147639500, real_size = 40582496 2024/11/23-17:26:53.974223 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.48 2024/11/23-17:26:53.974540 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:26:54.158253 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272232.sst): second pass time = 0.94's, 139.913'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 230.312'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1826.913'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.247'MB/sec write SST data time = 0.05's, 802.636'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49097.103'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 139.913'MB/sec zip pipeline throughput = 139.913'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.97 avg-zval = 311.48 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.97 avg-zval = 311.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6471 value = 3.2649 dict = 1.00 all = 3.1644 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.246227 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938732 Billion total entry num = 0.052746461 Billion write speed all = 0.348343220 MB/sec (with seq num) write speed all = 0.334599084 MB/sec (without seq num) 2024/11/23-17:26:54.211321 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272232: 129433 keys, 42415059 bytes, temperature: kWarm 2024/11/23-17:26:54.211387 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354014211355, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272232, "file_size": 42415059, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873554, "largest_seqno": 4146760353, "table_properties": {"data_size": 40316304, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629509, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354012, "oldest_key_time": 0, "file_creation_time": 1732354012, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272232, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:26:54.254540 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst) /dev/shm/ToplingTemp/Topling-132551-5ofXB6 2024/11/23-17:26:55.013212 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst): first pass time = 0.76's, 176.925'MB/sec 2024/11/23-17:26:57.028076 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst): old prealloc_size = 147639500, real_size = 40869872 2024/11/23-17:26:57.028229 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.77 2024/11/23-17:26:57.028415 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:26:57.292362 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272233.sst): second pass time = 2.01's, 65.352'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.26's, 161.596'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1734.757'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.164'MB/sec write SST data time = 0.07's, 638.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36220.058'MB/sec zip my value time = 2.01's, unzip length = 0.132'GB zip my value throughput = 65.352'MB/sec zip pipeline throughput = 118.146'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.97 avg-zval = 313.77 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.97 avg-zval = 313.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9024 value = 3.2411 dict = 1.00 all = 3.1431 } Zip/UnZip{ index = 0.1449 value = 0.3085 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.378763 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068166 Billion total entry num = 0.052875895 Billion write speed all = 0.352708394 MB/sec (with seq num) write speed all = 0.338931937 MB/sec (without seq num) 2024/11/23-17:26:57.348687 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272233: 129434 keys, 42702482 bytes, temperature: kWarm 2024/11/23-17:26:57.348776 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354017348728, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272233, "file_size": 42702482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4146755054, "table_properties": {"data_size": 40612320, "index_size": 225024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131630010, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354014, "oldest_key_time": 0, "file_creation_time": 1732354014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272233, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:26:57.393359 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst) /dev/shm/ToplingTemp/Topling-132551-6OyFr7 2024/11/23-17:26:58.036702 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst): first pass time = 0.64's, 208.639'MB/sec 2024/11/23-17:26:58.943564 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst): old prealloc_size = 147639500, real_size = 40584272 2024/11/23-17:26:58.943688 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.50 2024/11/23-17:26:58.943859 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:26:59.154414 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272234.sst): second pass time = 0.91's, 145.259'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 201.211'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2339.541'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.225'MB/sec write SST data time = 0.07's, 632.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 30825.888'MB/sec zip my value time = 0.91's, unzip length = 0.132'GB zip my value throughput = 145.259'MB/sec zip pipeline throughput = 145.259'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 311.50 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 311.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6470 value = 3.2648 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197596 Billion total entry num = 0.053005325 Billion write speed all = 0.357087441 MB/sec (with seq num) write speed all = 0.343278100 MB/sec (without seq num) 2024/11/23-17:26:59.214195 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272234: 129430 keys, 42416987 bytes, temperature: kWarm 2024/11/23-17:26:59.214285 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354019214237, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272234, "file_size": 42416987, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4146760485, "table_properties": {"data_size": 40318080, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629304, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354017, "oldest_key_time": 0, "file_creation_time": 1732354017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272234, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:26:59.290752 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst) /dev/shm/ToplingTemp/Topling-132551-lMzdz4 2024/11/23-17:26:59.991036 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst): first pass time = 0.70's, 191.680'MB/sec 2024/11/23-17:27:00.804011 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst): old prealloc_size = 147639500, real_size = 40587360 2024/11/23-17:27:00.804137 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.52 2024/11/23-17:27:00.804301 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:00.970966 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272235.sst): second pass time = 0.81's, 162.061'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.069'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2184.470'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 892.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 53802.623'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.061'MB/sec zip pipeline throughput = 162.061'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.98 avg-zval = 311.52 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.98 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6471 value = 3.2645 dict = 1.00 all = 3.1640 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327028 Billion total entry num = 0.053134757 Billion write speed all = 0.361466513 MB/sec (with seq num) write speed all = 0.347624271 MB/sec (without seq num) 2024/11/23-17:27:01.023995 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272235: 129432 keys, 42420319 bytes, temperature: kWarm 2024/11/23-17:27:01.024084 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354021024034, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272235, "file_size": 42420319, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4146756884, "table_properties": {"data_size": 40321168, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629870, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354019, "oldest_key_time": 0, "file_creation_time": 1732354019, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272235, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:01.064527 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst) /dev/shm/ToplingTemp/Topling-132551-2VODj4 2024/11/23-17:27:01.992987 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst): first pass time = 0.93's, 144.570'MB/sec 2024/11/23-17:27:02.885671 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst): old prealloc_size = 147639500, real_size = 41081664 2024/11/23-17:27:02.885800 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.01 avg-zval = 315.44 2024/11/23-17:27:02.885959 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:03.048473 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272236.sst): second pass time = 0.89's, 147.581'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.620'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2235.836'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.05's, 947.184'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45231.529'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 147.581'MB/sec zip pipeline throughput = 147.581'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 315.44 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.01 avg-zval = 315.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9855 value = 3.2241 dict = 1.00 all = 3.1276 } Zip/UnZip{ index = 0.1432 value = 0.3102 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 9.776370 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456456 Billion total entry num = 0.053264185 Billion write speed all = 0.365841965 MB/sec (with seq num) write speed all = 0.351966943 MB/sec (without seq num) 2024/11/23-17:27:03.127881 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272236: 129428 keys, 42914516 bytes, temperature: kWarm 2024/11/23-17:27:03.127941 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354023127904, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272236, "file_size": 42914516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237882779, "largest_seqno": 4146696142, "table_properties": {"data_size": 40826800, "index_size": 222336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630166, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354021, "oldest_key_time": 0, "file_creation_time": 1732354021, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272236, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:03.173776 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst) /dev/shm/ToplingTemp/Topling-132551-I6knk4 2024/11/23-17:27:03.738019 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst): first pass time = 0.56's, 237.891'MB/sec 2024/11/23-17:27:04.600881 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst): old prealloc_size = 147639500, real_size = 40881664 2024/11/23-17:27:04.601022 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.00 avg-zval = 313.87 2024/11/23-17:27:04.601187 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:04.765218 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272237.sst): second pass time = 0.86's, 152.680'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.957'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1884.530'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 918.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46622.078'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.680'MB/sec zip pipeline throughput = 152.680'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.00 avg-zval = 313.87 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1024.00 avg-zval = 313.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9081 value = 3.2402 dict = 1.00 all = 3.1422 } Zip/UnZip{ index = 0.1448 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.908906 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585886 Billion total entry num = 0.053393615 Billion write speed all = 0.370221161 MB/sec (with seq num) write speed all = 0.356313201 MB/sec (without seq num) 2024/11/23-17:27:04.824276 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272237: 129430 keys, 42714549 bytes, temperature: kWarm 2024/11/23-17:27:04.824345 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354024824311, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272237, "file_size": 42714549, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4146732727, "table_properties": {"data_size": 40624304, "index_size": 224832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629715, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354023, "oldest_key_time": 0, "file_creation_time": 1732354023, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272237, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:04.864352 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst) /dev/shm/ToplingTemp/Topling-132551-WxVZB3 2024/11/23-17:27:05.607007 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst): first pass time = 0.74's, 180.738'MB/sec 2024/11/23-17:27:06.533376 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst): old prealloc_size = 147639500, real_size = 40976208 2024/11/23-17:27:06.533509 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.99 avg-zval = 314.61 2024/11/23-17:27:06.533667 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:06.694154 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272238.sst): second pass time = 0.93's, 142.189'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2067.440'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 1002.814'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46258.308'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 142.189'MB/sec zip pipeline throughput = 142.189'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.99 avg-zval = 314.61 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.99 avg-zval = 314.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9537 value = 3.2325 dict = 1.00 all = 3.1353 } Zip/UnZip{ index = 0.1438 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 10.041442 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715317 Billion total entry num = 0.053523046 Billion write speed all = 0.374597278 MB/sec (with seq num) write speed all = 0.360656480 MB/sec (without seq num) 2024/11/23-17:27:06.751496 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272238: 129431 keys, 42809010 bytes, temperature: kWarm 2024/11/23-17:27:06.751593 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354026751541, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272238, "file_size": 42809010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237912377, "largest_seqno": 4146761570, "table_properties": {"data_size": 40720320, "index_size": 223360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629659, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354024, "oldest_key_time": 0, "file_creation_time": 1732354024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272238, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:06.809350 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst) /dev/shm/ToplingTemp/Topling-132551-0qtxp6 2024/11/23-17:27:07.550783 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst): first pass time = 0.74's, 181.036'MB/sec 2024/11/23-17:27:08.461659 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst): old prealloc_size = 147639500, real_size = 40584640 2024/11/23-17:27:08.461793 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.50 2024/11/23-17:27:08.461954 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:08.627078 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272239.sst): second pass time = 0.91's, 144.619'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2317.543'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 886.835'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45422.846'MB/sec zip my value time = 0.91's, unzip length = 0.132'GB zip my value throughput = 144.619'MB/sec zip pipeline throughput = 178.612'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.98 avg-zval = 311.50 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.98 avg-zval = 311.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6471 value = 3.2647 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844749 Billion total entry num = 0.053652478 Billion write speed all = 0.378972785 MB/sec (with seq num) write speed all = 0.364999154 MB/sec (without seq num) 2024/11/23-17:27:08.682860 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272239: 129432 keys, 42417643 bytes, temperature: kWarm 2024/11/23-17:27:08.682925 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354028682890, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272239, "file_size": 42417643, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4146760124, "table_properties": {"data_size": 40318448, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629291, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354026, "oldest_key_time": 0, "file_creation_time": 1732354026, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272239, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:08.721695 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst) /dev/shm/ToplingTemp/Topling-132551-2Y9sX3 2024/11/23-17:27:09.465447 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst): first pass time = 0.74's, 180.469'MB/sec 2024/11/23-17:27:10.277750 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst): old prealloc_size = 147639500, real_size = 40585600 2024/11/23-17:27:10.277878 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.52 2024/11/23-17:27:10.278058 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:10.441347 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272240.sst): second pass time = 0.81's, 162.180'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2188.869'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 945.466'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48267.026'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.180'MB/sec zip pipeline throughput = 162.180'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 311.52 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6470 value = 3.2647 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1504 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.306513 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974179 Billion total entry num = 0.053781908 Billion write speed all = 0.383349233 MB/sec (with seq num) write speed all = 0.369342719 MB/sec (without seq num) 2024/11/23-17:27:10.497694 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272240: 129430 keys, 42418117 bytes, temperature: kWarm 2024/11/23-17:27:10.497769 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354030497731, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272240, "file_size": 42418117, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4146753805, "table_properties": {"data_size": 40319408, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629661, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354028, "oldest_key_time": 0, "file_creation_time": 1732354028, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272240, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:10.533917 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst) /dev/shm/ToplingTemp/Topling-132551-XXxho6 2024/11/23-17:27:11.117134 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst): first pass time = 0.58's, 230.149'MB/sec 2024/11/23-17:27:11.977748 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst): old prealloc_size = 147639500, real_size = 40944560 2024/11/23-17:27:11.977864 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.00 avg-zval = 314.34 2024/11/23-17:27:11.978060 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:12.142665 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272241.sst): second pass time = 0.86's, 153.066'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.427'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2259.627'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 926.778'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 41164.830'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 153.066'MB/sec zip pipeline throughput = 153.066'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 314.34 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1024.00 avg-zval = 314.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8573 value = 3.2353 dict = 1.00 all = 3.1376 } Zip/UnZip{ index = 0.1458 value = 0.3091 dict = 1.00 all = 0.3187 } ---------------------------- total value len = 10.439049 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103609 Billion total entry num = 0.053911338 Billion write speed all = 0.387726596 MB/sec (with seq num) write speed all = 0.373687152 MB/sec (without seq num) 2024/11/23-17:27:12.195859 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272241: 129430 keys, 42777928 bytes, temperature: kWarm 2024/11/23-17:27:12.195920 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354032195887, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272241, "file_size": 42777928, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4146760598, "table_properties": {"data_size": 40685536, "index_size": 226496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629872, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354030, "oldest_key_time": 0, "file_creation_time": 1732354030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272241, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:12.239018 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst) /dev/shm/ToplingTemp/Topling-132551-nYV5L5 2024/11/23-17:27:13.126210 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst): first pass time = 0.89's, 151.292'MB/sec 2024/11/23-17:27:13.926857 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst): old prealloc_size = 147639500, real_size = 40805392 2024/11/23-17:27:13.926999 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.04 avg-zval = 313.26 2024/11/23-17:27:13.927151 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:14.089350 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272242.sst): second pass time = 0.80's, 164.554'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1896.606'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 995.449'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 48393.843'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.554'MB/sec zip pipeline throughput = 164.554'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.04 avg-zval = 313.26 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1024.04 avg-zval = 313.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129425 bytes = 485360 } seq expand size = 905975 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7615 value = 3.2467 dict = 1.00 all = 3.1478 } Zip/UnZip{ index = 0.1479 value = 0.3080 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 10.571584 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233034 Billion total entry num = 0.054040763 Billion write speed all = 0.392100341 MB/sec (with seq num) write speed all = 0.378028084 MB/sec (without seq num) 2024/11/23-17:27:14.144822 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272242: 129425 keys, 42638642 bytes, temperature: kWarm 2024/11/23-17:27:14.144891 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354034144856, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272242, "file_size": 42638642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4146753806, "table_properties": {"data_size": 40543168, "index_size": 229696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629948, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354032, "oldest_key_time": 0, "file_creation_time": 1732354032, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272242, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:14.183268 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst) /dev/shm/ToplingTemp/Topling-132551-d5rxq6 2024/11/23-17:27:14.917383 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst): first pass time = 0.73's, 182.837'MB/sec 2024/11/23-17:27:15.710753 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst): old prealloc_size = 147639500, real_size = 40547568 2024/11/23-17:27:15.710874 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.28 2024/11/23-17:27:15.711013 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:15.871326 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272243.sst): second pass time = 0.79's, 166.052'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.763'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2298.853'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 972.512'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44606.066'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 166.052'MB/sec zip pipeline throughput = 166.052'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 311.28 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1020.91 avg-zval = 311.28 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 72311 bytes = 271184 } seq expand size = 506177 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.8495 value = 3.2672 dict = 1.00 all = 3.1815 } Zip/UnZip{ index = 0.1460 value = 0.3061 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 10.703720 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362463 Billion total entry num = 0.054170192 Billion write speed all = 0.396462628 MB/sec (with seq num) write speed all = 0.382357486 MB/sec (without seq num) 2024/11/23-17:27:15.924109 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272243: 129429 keys, 42186596 bytes, temperature: kWarm 2024/11/23-17:27:15.924189 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354035924146, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272243, "file_size": 42186596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146749684, "table_properties": {"data_size": 40288288, "index_size": 226752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629355, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354034, "oldest_key_time": 0, "file_creation_time": 1732354034, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:15.964572 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst) /dev/shm/ToplingTemp/Topling-132551-KeBdA3 2024/11/23-17:27:16.553273 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst): first pass time = 0.59's, 228.006'MB/sec 2024/11/23-17:27:17.511846 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst): old prealloc_size = 147639500, real_size = 40876288 2024/11/23-17:27:17.511968 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.98 avg-zval = 313.86 2024/11/23-17:27:17.512105 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:17.672535 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272244.sst): second pass time = 0.96's, 137.421'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.901'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1793.371'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 1016.075'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47897.185'MB/sec zip my value time = 0.96's, unzip length = 0.132'GB zip my value throughput = 137.421'MB/sec zip pipeline throughput = 137.421'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.98 avg-zval = 313.86 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1021.53 avg-zval = 313.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 84135 bytes = 315520 } seq expand size = 588945 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 7.0466 value = 3.2402 dict = 1.00 all = 3.1537 } Zip/UnZip{ index = 0.1419 value = 0.3086 dict = 1.00 all = 0.3171 } ---------------------------- total value len = 10.835938 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021176 GB avg = 0.015 KB total ukey num = 0.067491895 Billion total entry num = 0.054299624 Billion write speed all = 0.400826858 MB/sec (with seq num) write speed all = 0.386688843 MB/sec (without seq num) 2024/11/23-17:27:17.726161 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272244: 129432 keys, 42559505 bytes, temperature: kWarm 2024/11/23-17:27:17.726228 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354037726195, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272244, "file_size": 42559505, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146723221, "table_properties": {"data_size": 40623344, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629506, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354035, "oldest_key_time": 0, "file_creation_time": 1732354035, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:17.763606 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst) /dev/shm/ToplingTemp/Topling-132551-CUumG4 2024/11/23-17:27:18.484008 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst): first pass time = 0.72's, 186.319'MB/sec 2024/11/23-17:27:19.272544 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst): old prealloc_size = 147639500, real_size = 40721536 2024/11/23-17:27:19.272663 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.01 avg-zval = 312.64 2024/11/23-17:27:19.272785 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:19.433772 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272245.sst): second pass time = 0.79's, 167.076'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.917'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2342.637'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 955.586'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44695.212'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.076'MB/sec zip pipeline throughput = 167.076'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 312.64 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1021.25 avg-zval = 312.64 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 78317 bytes = 293696 } seq expand size = 548219 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.9257 value = 3.2529 dict = 1.00 all = 3.1668 } Zip/UnZip{ index = 0.1444 value = 0.3074 dict = 1.00 all = 0.3158 } ---------------------------- total value len = 10.968116 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621323 Billion total entry num = 0.054429052 Billion write speed all = 0.405189778 MB/sec (with seq num) write speed all = 0.391018876 MB/sec (without seq num) 2024/11/23-17:27:19.488897 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272245: 129428 keys, 42382791 bytes, temperature: kWarm 2024/11/23-17:27:19.488966 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354039488931, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272245, "file_size": 42382791, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146757380, "table_properties": {"data_size": 40464752, "index_size": 224256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629567, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354037, "oldest_key_time": 0, "file_creation_time": 1732354037, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:19.524406 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst) /dev/shm/ToplingTemp/Topling-132551-qKxvO3 2024/11/23-17:27:20.141429 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst): first pass time = 0.62's, 217.543'MB/sec 2024/11/23-17:27:21.112926 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst): old prealloc_size = 147639500, real_size = 41074592 2024/11/23-17:27:21.113050 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.39 2024/11/23-17:27:21.113201 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:21.274704 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272246.sst): second pass time = 0.97's, 135.580'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.202'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2052.101'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 997.834'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49790.762'MB/sec zip my value time = 0.97's, unzip length = 0.132'GB zip my value throughput = 135.580'MB/sec zip pipeline throughput = 135.580'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 315.39 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.60 avg-zval = 315.39 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 121524 bytes = 455728 } seq expand size = 850668 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9915 value = 3.2247 dict = 1.00 all = 3.1288 } Zip/UnZip{ index = 0.1430 value = 0.3101 dict = 1.00 all = 0.3196 } ---------------------------- total value len = 11.100597 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750749 Billion total entry num = 0.054558478 Billion write speed all = 0.409560995 MB/sec (with seq num) write speed all = 0.395357247 MB/sec (without seq num) 2024/11/23-17:27:21.330095 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272246: 129426 keys, 42897688 bytes, temperature: kWarm 2024/11/23-17:27:21.330162 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354041330130, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272246, "file_size": 42897688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146731253, "table_properties": {"data_size": 40819920, "index_size": 222144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630134, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354039, "oldest_key_time": 0, "file_creation_time": 1732354039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:21.370232 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst) /dev/shm/ToplingTemp/Topling-132551-wyfwE4 2024/11/23-17:27:21.915584 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst): first pass time = 0.55's, 246.129'MB/sec 2024/11/23-17:27:22.742853 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst): old prealloc_size = 147639500, real_size = 40551184 2024/11/23-17:27:22.742972 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.30 2024/11/23-17:27:22.743100 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:22.903896 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272247.sst): second pass time = 0.83's, 159.269'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2073.094'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 980.867'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45577.604'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.269'MB/sec zip pipeline throughput = 159.269'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.00 avg-zval = 311.30 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1021.17 avg-zval = 311.30 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 77159 bytes = 289360 } seq expand size = 540113 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.8111 value = 3.2670 dict = 1.00 all = 3.1799 } Zip/UnZip{ index = 0.1468 value = 0.3061 dict = 1.00 all = 0.3145 } ---------------------------- total value len = 11.232767 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880178 Billion total entry num = 0.054687907 Billion write speed all = 0.413924412 MB/sec (with seq num) write speed all = 0.399687724 MB/sec (without seq num) 2024/11/23-17:27:22.955699 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272247: 129429 keys, 42208337 bytes, temperature: kWarm 2024/11/23-17:27:22.955767 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354042955732, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272247, "file_size": 42208337, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146747510, "table_properties": {"data_size": 40290624, "index_size": 228032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629373, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354041, "oldest_key_time": 0, "file_creation_time": 1732354041, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:22.995854 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst) /dev/shm/ToplingTemp/Topling-132551-C3sEW3 2024/11/23-17:27:23.497336 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst): first pass time = 0.50's, 267.666'MB/sec 2024/11/23-17:27:24.518214 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst): old prealloc_size = 147639500, real_size = 40583104 2024/11/23-17:27:24.518331 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.01 avg-zval = 311.59 2024/11/23-17:27:24.518428 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:24.694768 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272248.sst): second pass time = 1.02's, 129.040'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 238.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2340.789'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.251'MB/sec write SST data time = 0.05's, 880.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37085.341'MB/sec zip my value time = 1.02's, unzip length = 0.132'GB zip my value throughput = 129.040'MB/sec zip pipeline throughput = 196.435'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 311.59 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.68 avg-zval = 311.59 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 31012 bytes = 116304 } seq expand size = 217084 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9936 value = 3.2639 dict = 1.00 all = 3.1906 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3134 } ---------------------------- total value len = 11.364614 GB avg = 0.207 KB (by entry num) total key len = 1.490005 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009607 Billion total entry num = 0.054817336 Billion write speed all = 0.418274678 MB/sec (with seq num) write speed all = 0.404005127 MB/sec (without seq num) 2024/11/23-17:27:24.747090 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272248: 129429 keys, 42067067 bytes, temperature: kWarm 2024/11/23-17:27:24.747160 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354044747125, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272248, "file_size": 42067067, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146744850, "table_properties": {"data_size": 40328496, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630071, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354042, "oldest_key_time": 0, "file_creation_time": 1732354042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:24.785265 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst) /dev/shm/ToplingTemp/Topling-132551-VURGg7 2024/11/23-17:27:25.296989 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst): first pass time = 0.51's, 262.311'MB/sec 2024/11/23-17:27:26.079559 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst): old prealloc_size = 147639500, real_size = 40496000 2024/11/23-17:27:26.079678 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.91 2024/11/23-17:27:26.079778 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:26.240035 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272249.sst): second pass time = 0.78's, 168.503'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.782'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2282.906'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 1003.919'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47562.116'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.503'MB/sec zip pipeline throughput = 168.503'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.91 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.04 avg-zval = 310.91 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 37639 bytes = 141168 } seq expand size = 263473 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9655 value = 3.2711 dict = 1.00 all = 3.1953 } Zip/UnZip{ index = 0.1436 value = 0.3057 dict = 1.00 all = 0.3130 } ---------------------------- total value len = 11.496507 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139036 Billion total entry num = 0.054946765 Billion write speed all = 0.422629310 MB/sec (with seq num) write speed all = 0.408326787 MB/sec (without seq num) 2024/11/23-17:27:26.292284 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272249: 129429 keys, 42005134 bytes, temperature: kWarm 2024/11/23-17:27:26.292351 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354046292318, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272249, "file_size": 42005134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146753725, "table_properties": {"data_size": 40240496, "index_size": 222976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629671, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354044, "oldest_key_time": 0, "file_creation_time": 1732354044, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:26.333930 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst) /dev/shm/ToplingTemp/Topling-132551-xch6L6 2024/11/23-17:27:26.949067 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst): first pass time = 0.62's, 218.206'MB/sec 2024/11/23-17:27:27.751239 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst): old prealloc_size = 147639500, real_size = 40477008 2024/11/23-17:27:27.751360 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.76 2024/11/23-17:27:27.751458 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:27.981455 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272250.sst): second pass time = 0.80's, 164.250'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 182.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2254.776'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.193'MB/sec write SST data time = 0.06's, 668.000'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46238.634'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.250'MB/sec zip pipeline throughput = 164.250'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.76 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.89 avg-zval = 310.76 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 35230 bytes = 132128 } seq expand size = 246610 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9737 value = 3.2726 dict = 1.00 all = 3.1974 } Zip/UnZip{ index = 0.1434 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.628383 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268467 Billion total entry num = 0.055076196 Billion write speed all = 0.426980243 MB/sec (with seq num) write speed all = 0.412644841 MB/sec (without seq num) 2024/11/23-17:27:28.038724 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272250: 129431 keys, 41976897 bytes, temperature: kWarm 2024/11/23-17:27:28.038815 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354048038767, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272250, "file_size": 41976897, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146746274, "table_properties": {"data_size": 40221760, "index_size": 222720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629986, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354046, "oldest_key_time": 0, "file_creation_time": 1732354046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:28.092675 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst) /dev/shm/ToplingTemp/Topling-132551-GfavP4 2024/11/23-17:27:28.725645 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst): first pass time = 0.63's, 212.065'MB/sec 2024/11/23-17:27:29.562296 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst): old prealloc_size = 147639500, real_size = 41122224 2024/11/23-17:27:29.562430 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0409 GB avg-val =1016.97 avg-zval = 315.75 2024/11/23-17:27:29.562590 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:27:29.730545 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272251.sst): second pass time = 0.84's, 157.470'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1965.282'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.263'MB/sec write SST data time = 0.05's, 950.758'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39912.765'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 157.470'MB/sec zip pipeline throughput = 157.470'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.97 avg-zval = 315.75 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.71 avg-zval = 315.75 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 124636 bytes = 467392 } seq expand size = 872452 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0409 GB dict = 1.38 MB all = 0.0430 GB } UnZip/Zip{ index = 7.0283 value = 3.2208 dict = 1.00 all = 3.1245 } Zip/UnZip{ index = 0.1423 value = 0.3105 dict = 1.00 all = 0.3201 } ---------------------------- total value len = 11.760885 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032048 GB avg = 0.015 KB total ukey num = 0.068397900 Billion total entry num = 0.055205629 Billion write speed all = 0.431350923 MB/sec (with seq num) write speed all = 0.416982650 MB/sec (without seq num) 2024/11/23-17:27:29.784685 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272251: 129433 keys, 42956956 bytes, temperature: kWarm 2024/11/23-17:27:29.784755 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354049784720, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272251, "file_size": 42956956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4146761390, "table_properties": {"data_size": 40868704, "index_size": 220992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629584, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732354048, "oldest_key_time": 0, "file_creation_time": 1732354048, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:29.822614 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272252.sst) /dev/shm/ToplingTemp/Topling-132551-5CnUJ4 2024/11/23-17:27:29.831914 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272252.sst): first pass time = 0.01's, 315.370'MB/sec 2024/11/23-17:27:29.864237 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272252.sst): old prealloc_size = 147639500, real_size = 930048 2024/11/23-17:27:29.864288 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272252.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2816 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.34 raw-val = 0.0029 GB zip-val = 0.0009 GB avg-val =1017.10 avg-zval = 329.61 2024/11/23-17:27:29.867971 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00/272252.sst): second pass time = 0.03's, 90.291'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 261.152'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 30.735'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.299'MB/sec write SST data time = 0.00's, 727.000'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.033'MB, throughput = 1318.191'MB/sec zip my value time = 0.03's, unzip length = 0.003'GB zip my value throughput = 90.291'MB/sec zip pipeline throughput = 90.291'MB/sec entries = 2816 avg-key = 20.00 avg-zkey = 0.34 avg-val = 1017.10 avg-zval = 329.61 usrkeys = 2816 avg-key = 12.00 avg-zkey = 0.34 avg-val = 1024.10 avg-zval = 329.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2816 bytes = 9520 } seq expand size = 19712 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0000 GB value = 0.0029 GB dict = 0.03 MB all = 0.0029 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.03 MB all = 0.0010 GB } UnZip/Zip{ index = 35.2000 value = 3.0858 dict = 1.00 all = 3.0035 } Zip/UnZip{ index = 0.0284 value = 0.3241 dict = 1.00 all = 0.3329 } ---------------------------- total value len = 11.763769 GB avg = 0.213 KB (by entry num) total key len = 1.497827 GB avg = 0.027 KB total ukey len = 1.032081 GB avg = 0.015 KB total ukey num = 0.068400716 Billion total entry num = 0.055208445 Billion write speed all = 0.431444648 MB/sec (with seq num) write speed all = 0.417075706 MB/sec (without seq num) 2024/11/23-17:27:29.878309 139957257127488 [compaction_job.cc:1948] [default] [JOB 31903] Generated table #272252: 2816 keys, 972341 bytes, temperature: kWarm 2024/11/23-17:27:29.878371 139957257127488 EVENT_LOG_v1 {"time_micros": 1732354049878340, "cf_name": "default", "job": 31903, "event": "table_file_creation", "file_number": 272252, "file_size": 972341, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4027849193, "largest_seqno": 4145240819, "table_properties": {"data_size": 928176, "index_size": 960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 56320, "raw_average_key_size": 20, "raw_value_size": 2864140, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 2816, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732354049, "oldest_key_time": 0, "file_creation_time": 1732354049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:27:29.890084 139957257127488 [compaction_job.cc:703] job-31903: subcompact[0], size: 0.935982 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31903/att-00 [272230,272231,272232,272233,272234,272235,272236,272237,272238,272239,272240,272241,272242,272243,272244,272245,272246,272247,272248,272249,272250,272251,272252] 2024/11/23-17:27:30.042310 139957257127488 (Original Log Time 2024/11/23-17:26:47.195617) EVENT_LOG_v1 {"time_micros": 1732354007195574, "job": 31903, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [272224], "files_L3": [272143, 272144, 272145, 272146, 272147, 272148, 272149, 272150, 272151, 272152, 272153, 272154, 272155, 272156, 272157, 272158, 272159, 272160, 272161, 272162, 272163, 272164], "score": 0.994032, "input_data_size": 967721563, "oldest_snapshot_seqno": 4146795821} 2024/11/23-17:27:30.042314 139957257127488 (Original Log Time 2024/11/23-17:27:30.040609) [compaction_job.cc:2022] [default] [JOB 31903] Compacted 1@2 + 22@3 files to L3 => 935982440 bytes 2024/11/23-17:27:30.042315 139957257127488 (Original Log Time 2024/11/23-17:27:30.042230) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 22.7 rd, 21.9 wr, level 3, files in(1, 22) out(23 +0 blob) MB in(31.2, 891.7 +0.0 blob) out(892.6 +0.0 blob), read-write-amplify(58.2) write-amplify(28.6) OK, records in: 2879370, records dropped: 29089 output_compression: Snappy 2024/11/23-17:27:30.042319 139957257127488 (Original Log Time 2024/11/23-17:27:30.042282) EVENT_LOG_v1 {"time_micros": 1732354050042260, "cf": "default", "job": 31903, "event": "compaction_finished", "compaction_time_micros": 42694474, "compaction_time_cpu_micros": 8717992, "output_level": 3, "num_output_files": 23, "total_output_size": 935982440, "num_input_records": 2879370, "num_output_records": 2850281, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 773893675, "file_range_sync_nanos": 0, "file_fsync_nanos": 18229, "file_prepare_write_nanos": 50175, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:27:30.156688 139957257127488 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732354007,"executesMs":43003,"compactionJobId":31903,"attempt":0,"compactionInputRawBytes":2985890023,"compactionInputZipBytes":967721563,"compactionOutputRawBytes":0,"compactionOutputZipBytes":935982440,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f3476b4358087e292da8a0401b079226-039d83db8a90bd06-00"}}, response = 2024/11/23-17:27:30.156766 139957257127488 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31903/att-00: olev 3, work 43.003 s, result 8.367 ms, install 1.727 ms, input{raw 2.781 GiB zip 922.891 MiB}