2024/11/23-19:20:05.280956 139957475337792 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00: fork to child time = 0.007088 sec 2024/11/23-19:20:05.281723 139957475337792 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00 2024/11/23-19:20:05.289156 139957475337792 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00 2024/11/23-19:20:05.293992 139957475337792 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:20:05.294304 139957475337792 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:20:05.346132 139957475337792 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34047/att-00/MANIFEST-206347 2024/11/23-19:20:05.364259 139957475337792 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34047/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 276895, last_sequence is 4211732085, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 276767 2024/11/23-19:20:05.364267 139957475337792 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:20:05.364483 139957475337792 [version_set.cc:5721] Creating manifest 276895 2024/11/23-19:20:05.398430 139957475337792 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:20:05.423452 139957475337792 [compaction_job.cc:2437] [default] [JOB 34047] Compacting 1@2 + 23@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-19:20:05.423473 139957475337792 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25489 Base level 2, inputs: [276892(31MB)], [276821(40MB) 276822(40MB) 276823(40MB) 276824(40MB) 276825(40MB) 276826(40MB) 276827(40MB) 276828(40MB) 276829(40MB) 276830(40MB) 276831(40MB) 276832(40MB) 276833(40MB) 276834(40MB) 276835(40MB) 276836(40MB) 276837(40MB) 276838(40MB) 276839(40MB) 276840(40MB) 276841(40MB) 276842(40MB) 276843(897KB)] 2024/11/23-19:20:05.431258 139957475337792 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:20:05.431289 139957475337792 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:20:05.431306 139957475337792 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:20:05.431312 139957475337792 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:20:05.431325 139957475337792 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:20:06.139022 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst) /dev/shm/ToplingTemp/Topling-163666-GSUYRS 2024/11/23-19:20:06.328757 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst): first pass time = 0.19's, 707.552'MB/sec 2024/11/23-19:20:07.226867 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst): old prealloc_size = 147639500, real_size = 40589520 2024/11/23-19:20:07.227028 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.55 2024/11/23-19:20:07.227203 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:07.398422 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276896.sst): second pass time = 0.89's, 147.242'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 247.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1783.801'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 840.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41381.592'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 147.242'MB/sec zip pipeline throughput = 148.319'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 311.55 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2643 dict = 1.00 all = 3.1638 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679865 Billion total entry num = 0.052487594 Billion write speed all = 0.278024098 MB/sec (with seq num) write speed all = 0.266825306 MB/sec (without seq num) 2024/11/23-19:20:07.451839 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276896: 129430 keys, 42422294 bytes, temperature: kWarm 2024/11/23-19:20:07.451961 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360807451884, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276896, "file_size": 42422294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4211712372, "table_properties": {"data_size": 40324288, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629159, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360806, "oldest_key_time": 0, "file_creation_time": 1732360806, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276896, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:07.503126 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst) /dev/shm/ToplingTemp/Topling-163666-FdTXfT 2024/11/23-19:20:08.064753 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst): first pass time = 0.56's, 239.000'MB/sec 2024/11/23-19:20:10.863132 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst): old prealloc_size = 147639500, real_size = 40867680 2024/11/23-19:20:10.863260 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.76 2024/11/23-19:20:10.863423 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:11.033058 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276897.sst): second pass time = 2.80's, 47.051'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 251.320'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2029.826'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.05's, 885.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50450.379'MB/sec zip my value time = 2.80's, unzip length = 0.132'GB zip my value throughput = 47.051'MB/sec zip pipeline throughput = 108.262'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.96 avg-zval = 313.76 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.96 avg-zval = 313.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9201 value = 3.2413 dict = 1.00 all = 3.1432 } Zip/UnZip{ index = 0.1445 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809299 Billion total entry num = 0.052617028 Billion write speed all = 0.281600577 MB/sec (with seq num) write speed all = 0.270375258 MB/sec (without seq num) 2024/11/23-19:20:11.086256 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276897: 129434 keys, 42700584 bytes, temperature: kWarm 2024/11/23-19:20:11.086318 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360811086286, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276897, "file_size": 42700584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4211690159, "table_properties": {"data_size": 40610704, "index_size": 224448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629553, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360807, "oldest_key_time": 0, "file_creation_time": 1732360807, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276897, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:11.128428 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst) /dev/shm/ToplingTemp/Topling-163666-FfgnfP 2024/11/23-19:20:11.819826 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst): first pass time = 0.69's, 194.136'MB/sec 2024/11/23-19:20:12.738493 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst): old prealloc_size = 147639500, real_size = 40575936 2024/11/23-19:20:12.738639 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.44 2024/11/23-19:20:12.738796 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:12.910650 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276898.sst): second pass time = 0.92's, 143.399'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2218.366'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 823.587'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43159.357'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 143.399'MB/sec zip pipeline throughput = 144.202'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.44 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6764 value = 3.2654 dict = 1.00 all = 3.1649 } Zip/UnZip{ index = 0.1498 value = 0.3062 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938732 Billion total entry num = 0.052746461 Billion write speed all = 0.285189725 MB/sec (with seq num) write speed all = 0.273937355 MB/sec (without seq num) 2024/11/23-19:20:12.963730 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276898: 129433 keys, 42408452 bytes, temperature: kWarm 2024/11/23-19:20:12.963799 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360812963765, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276898, "file_size": 42408452, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873554, "largest_seqno": 4211705049, "table_properties": {"data_size": 40310768, "index_size": 232640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629563, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360811, "oldest_key_time": 0, "file_creation_time": 1732360811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276898, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:13.008172 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst) /dev/shm/ToplingTemp/Topling-163666-8KrFLS 2024/11/23-19:20:13.868393 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst): first pass time = 0.86's, 156.037'MB/sec 2024/11/23-19:20:16.023429 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst): old prealloc_size = 147639500, real_size = 40872576 2024/11/23-19:20:16.023578 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.79 2024/11/23-19:20:16.023768 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:16.327678 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276899.sst): second pass time = 2.15's, 61.102'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.30's, 140.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1770.923'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.150'MB/sec write SST data time = 0.09's, 485.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29590.979'MB/sec zip my value time = 2.15's, unzip length = 0.132'GB zip my value throughput = 61.102'MB/sec zip pipeline throughput = 139.283'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.96 avg-zval = 313.79 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.96 avg-zval = 313.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9162 value = 3.2409 dict = 1.00 all = 3.1429 } Zip/UnZip{ index = 0.1446 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068167 Billion total entry num = 0.052875896 Billion write speed all = 0.288766666 MB/sec (with seq num) write speed all = 0.277487711 MB/sec (without seq num) 2024/11/23-19:20:16.384934 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276899: 129435 keys, 42705766 bytes, temperature: kWarm 2024/11/23-19:20:16.385038 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360816384988, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276899, "file_size": 42705766, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4211707274, "table_properties": {"data_size": 40615472, "index_size": 224576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629746, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360812, "oldest_key_time": 0, "file_creation_time": 1732360812, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276899, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:16.438446 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst) /dev/shm/ToplingTemp/Topling-163666-72BTRQ 2024/11/23-19:20:17.043124 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst): first pass time = 0.60's, 221.987'MB/sec 2024/11/23-19:20:18.577509 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst): old prealloc_size = 147639500, real_size = 40586768 2024/11/23-19:20:18.577633 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.53 2024/11/23-19:20:18.577794 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:18.742320 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276900.sst): second pass time = 1.53's, 85.872'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.417'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2253.696'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 901.903'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 26896.153'MB/sec zip my value time = 1.53's, unzip length = 0.132'GB zip my value throughput = 85.872'MB/sec zip pipeline throughput = 123.218'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.98 avg-zval = 311.53 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.98 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6763 value = 3.2645 dict = 1.00 all = 3.1640 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197599 Billion total entry num = 0.053005328 Billion write speed all = 0.292350749 MB/sec (with seq num) write speed all = 0.281044913 MB/sec (without seq num) 2024/11/23-19:20:18.834971 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276900: 129432 keys, 42419667 bytes, temperature: kWarm 2024/11/23-19:20:18.835050 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360818835017, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276900, "file_size": 42419667, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4211714373, "table_properties": {"data_size": 40321600, "index_size": 232640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629120, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360816, "oldest_key_time": 0, "file_creation_time": 1732360816, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276900, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:18.873843 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst) /dev/shm/ToplingTemp/Topling-163666-wyHjSR 2024/11/23-19:20:19.498194 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst): first pass time = 0.62's, 214.985'MB/sec 2024/11/23-19:20:20.364099 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst): old prealloc_size = 147639500, real_size = 40584544 2024/11/23-19:20:20.364255 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.51 2024/11/23-19:20:20.364447 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:20.616736 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276901.sst): second pass time = 0.86's, 152.327'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 167.907'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1750.523'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.162'MB/sec write SST data time = 0.05's, 804.606'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44574.391'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.327'MB/sec zip pipeline throughput = 157.570'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.51 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2647 dict = 1.00 all = 3.1643 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.643833 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327029 Billion total entry num = 0.053134758 Billion write speed all = 0.295938652 MB/sec (with seq num) write speed all = 0.284605776 MB/sec (without seq num) 2024/11/23-19:20:20.673369 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276901: 129430 keys, 42417134 bytes, temperature: kWarm 2024/11/23-19:20:20.673439 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360820673404, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276901, "file_size": 42417134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4211713661, "table_properties": {"data_size": 40319312, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629883, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360818, "oldest_key_time": 0, "file_creation_time": 1732360818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276901, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:20.710363 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst) /dev/shm/ToplingTemp/Topling-163666-PEtA5S 2024/11/23-19:20:21.441163 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst): first pass time = 0.73's, 183.669'MB/sec 2024/11/23-19:20:22.329176 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst): old prealloc_size = 147639500, real_size = 40900960 2024/11/23-19:20:22.329304 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.00 avg-zval = 314.00 2024/11/23-19:20:22.329479 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:22.494830 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276902.sst): second pass time = 0.89's, 148.489'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 258.009'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2058.317'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 958.160'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45475.034'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 148.489'MB/sec zip pipeline throughput = 148.489'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 314.00 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1024.00 avg-zval = 314.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8457 value = 3.2388 dict = 1.00 all = 3.1408 } Zip/UnZip{ index = 0.1461 value = 0.3088 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458942 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456459 Billion total entry num = 0.053264188 Billion write speed all = 0.299526173 MB/sec (with seq num) write speed all = 0.288166261 MB/sec (without seq num) 2024/11/23-19:20:22.552507 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276902: 129430 keys, 42733650 bytes, temperature: kWarm 2024/11/23-19:20:22.552577 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360822552540, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276902, "file_size": 42733650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4211690541, "table_properties": {"data_size": 40641552, "index_size": 226880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630131, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360820, "oldest_key_time": 0, "file_creation_time": 1732360820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276902, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:22.591718 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst) /dev/shm/ToplingTemp/Topling-163666-Z4oJoQ 2024/11/23-19:20:23.215426 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst): first pass time = 0.62's, 215.206'MB/sec 2024/11/23-19:20:24.087200 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst): old prealloc_size = 147639500, real_size = 40979168 2024/11/23-19:20:24.087338 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.64 2024/11/23-19:20:24.087501 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:24.276546 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276903.sst): second pass time = 0.87's, 151.112'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 226.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2187.826'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.258'MB/sec write SST data time = 0.06's, 670.505'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41959.135'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 151.112'MB/sec zip pipeline throughput = 151.112'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.64 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9395 value = 3.2323 dict = 1.00 all = 3.1351 } Zip/UnZip{ index = 0.1441 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585886 Billion total entry num = 0.053393615 Billion write speed all = 0.303114098 MB/sec (with seq num) write speed all = 0.291727123 MB/sec (without seq num) 2024/11/23-19:20:24.336284 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276903: 129427 keys, 42811793 bytes, temperature: kWarm 2024/11/23-19:20:24.336377 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360824336328, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276903, "file_size": 42811793, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4211689822, "table_properties": {"data_size": 40722832, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629601, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360822, "oldest_key_time": 0, "file_creation_time": 1732360822, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276903, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:24.403857 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst) /dev/shm/ToplingTemp/Topling-163666-8DaL7Q 2024/11/23-19:20:25.151206 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst): first pass time = 0.75's, 179.605'MB/sec 2024/11/23-19:20:26.033175 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst): old prealloc_size = 147639500, real_size = 41031808 2024/11/23-19:20:26.033311 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.06 2024/11/23-19:20:26.033476 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:26.207153 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276904.sst): second pass time = 0.88's, 149.376'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.431'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2030.662'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.06's, 771.032'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48062.777'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.376'MB/sec zip pipeline throughput = 149.376'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.06 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.99 avg-zval = 315.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0342 value = 3.2279 dict = 1.00 all = 3.1312 } Zip/UnZip{ index = 0.1422 value = 0.3098 dict = 1.00 all = 0.3194 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715316 Billion total entry num = 0.053523045 Billion write speed all = 0.306700462 MB/sec (with seq num) write speed all = 0.295286471 MB/sec (without seq num) 2024/11/23-19:20:26.261060 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276904: 129430 keys, 42864695 bytes, temperature: kWarm 2024/11/23-19:20:26.261119 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360826261086, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276904, "file_size": 42864695, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4211714810, "table_properties": {"data_size": 40778480, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629413, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360824, "oldest_key_time": 0, "file_creation_time": 1732360824, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276904, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:26.318170 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst) /dev/shm/ToplingTemp/Topling-163666-c0r3tQ 2024/11/23-19:20:26.917361 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst): first pass time = 0.60's, 224.019'MB/sec 2024/11/23-19:20:27.775622 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst): old prealloc_size = 147639500, real_size = 40709920 2024/11/23-19:20:27.775745 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.49 2024/11/23-19:20:27.775911 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:27.938858 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276905.sst): second pass time = 0.86's, 153.499'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.651'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2171.281'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 932.089'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45655.603'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 153.499'MB/sec zip pipeline throughput = 153.499'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1016.97 avg-zval = 312.49 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1023.97 avg-zval = 312.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.7432 value = 3.2544 dict = 1.00 all = 3.1549 } Zip/UnZip{ index = 0.1483 value = 0.3073 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 10.173976 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844749 Billion total entry num = 0.053652478 Billion write speed all = 0.310288109 MB/sec (with seq num) write speed all = 0.298847045 MB/sec (without seq num) 2024/11/23-19:20:27.991471 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276905: 129433 keys, 42542968 bytes, temperature: kWarm 2024/11/23-19:20:27.991558 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360827991505, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276905, "file_size": 42542968, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4211714811, "table_properties": {"data_size": 40447056, "index_size": 230336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629613, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360826, "oldest_key_time": 0, "file_creation_time": 1732360826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276905, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:28.031077 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst) /dev/shm/ToplingTemp/Topling-163666-ddvwYP 2024/11/23-19:20:28.592406 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst): first pass time = 0.56's, 239.123'MB/sec 2024/11/23-19:20:29.363297 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst): old prealloc_size = 147639500, real_size = 40584192 2024/11/23-19:20:29.363425 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.51 2024/11/23-19:20:29.363596 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:29.525903 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276906.sst): second pass time = 0.77's, 170.915'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2195.466'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 974.403'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48233.844'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.915'MB/sec zip pipeline throughput = 170.915'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.51 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2647 dict = 1.00 all = 3.1643 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.306511 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974178 Billion total entry num = 0.053781907 Billion write speed all = 0.313876625 MB/sec (with seq num) write speed all = 0.302408446 MB/sec (without seq num) 2024/11/23-19:20:29.577704 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276906: 129429 keys, 42416823 bytes, temperature: kWarm 2024/11/23-19:20:29.577774 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360829577738, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276906, "file_size": 42416823, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873605, "largest_seqno": 4211714308, "table_properties": {"data_size": 40318960, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629321, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360828, "oldest_key_time": 0, "file_creation_time": 1732360828, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276906, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:29.615847 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst) /dev/shm/ToplingTemp/Topling-163666-sFw3oQ 2024/11/23-19:20:30.302835 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst): first pass time = 0.69's, 195.382'MB/sec 2024/11/23-19:20:31.142653 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst): old prealloc_size = 147639500, real_size = 40589648 2024/11/23-19:20:31.142775 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.56 2024/11/23-19:20:31.142939 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:31.303465 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276907.sst): second pass time = 0.84's, 156.973'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.833'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2334.323'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 978.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44926.024'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.973'MB/sec zip pipeline throughput = 156.973'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.56 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6744 value = 3.2643 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.439047 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103607 Billion total entry num = 0.053911336 Billion write speed all = 0.317463241 MB/sec (with seq num) write speed all = 0.305968007 MB/sec (without seq num) 2024/11/23-19:20:31.356005 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276907: 129429 keys, 42422386 bytes, temperature: kWarm 2024/11/23-19:20:31.356075 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360831356039, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276907, "file_size": 42422386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4211713222, "table_properties": {"data_size": 40324416, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629909, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360829, "oldest_key_time": 0, "file_creation_time": 1732360829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276907, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:31.398075 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst) /dev/shm/ToplingTemp/Topling-163666-t7kAPO 2024/11/23-19:20:32.193537 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst): first pass time = 0.80's, 168.738'MB/sec 2024/11/23-19:20:33.001465 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst): old prealloc_size = 147639500, real_size = 41086128 2024/11/23-19:20:33.001588 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 315.47 2024/11/23-19:20:33.001755 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:33.161105 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276908.sst): second pass time = 0.81's, 163.073'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.883'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2240.213'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 1022.067'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48447.757'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.073'MB/sec zip pipeline throughput = 163.073'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 315.47 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 315.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9916 value = 3.2237 dict = 1.00 all = 3.1272 } Zip/UnZip{ index = 0.1430 value = 0.3102 dict = 1.00 all = 0.3198 } ---------------------------- total value len = 10.571583 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233036 Billion total entry num = 0.054040765 Billion write speed all = 0.321048826 MB/sec (with seq num) write speed all = 0.309526565 MB/sec (without seq num) 2024/11/23-19:20:33.212188 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276908: 129429 keys, 42918975 bytes, temperature: kWarm 2024/11/23-19:20:33.212269 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360833212224, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276908, "file_size": 42918975, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237916889, "largest_seqno": 4211694258, "table_properties": {"data_size": 40831456, "index_size": 222144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629697, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360831, "oldest_key_time": 0, "file_creation_time": 1732360831, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276908, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:33.249218 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst) /dev/shm/ToplingTemp/Topling-163666-nduBHQ 2024/11/23-19:20:33.906517 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst): first pass time = 0.66's, 204.211'MB/sec 2024/11/23-19:20:34.680804 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst): old prealloc_size = 147639500, real_size = 40585168 2024/11/23-19:20:34.680927 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.53 2024/11/23-19:20:34.681108 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:34.843815 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276909.sst): second pass time = 0.77's, 170.170'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.253'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2303.977'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.05's, 933.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44631.709'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.170'MB/sec zip pipeline throughput = 170.170'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.02 avg-zval = 311.53 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.02 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6742 value = 3.2646 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1498 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.704119 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362463 Billion total entry num = 0.054170192 Billion write speed all = 0.324635578 MB/sec (with seq num) write speed all = 0.313086240 MB/sec (without seq num) 2024/11/23-19:20:34.896120 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276909: 129427 keys, 42417875 bytes, temperature: kWarm 2024/11/23-19:20:34.896211 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360834896160, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276909, "file_size": 42417875, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4211705268, "table_properties": {"data_size": 40319936, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630070, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360833, "oldest_key_time": 0, "file_creation_time": 1732360833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276909, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:34.946065 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst) /dev/shm/ToplingTemp/Topling-163666-lFy7aQ 2024/11/23-19:20:35.503531 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst): first pass time = 0.56's, 240.790'MB/sec 2024/11/23-19:20:36.444592 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst): old prealloc_size = 147639500, real_size = 40487616 2024/11/23-19:20:36.444711 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.84 2024/11/23-19:20:36.444809 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:36.603948 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276910.sst): second pass time = 0.94's, 139.976'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.622'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2273.614'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 1006.149'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39527.243'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 139.976'MB/sec zip pipeline throughput = 139.976'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.14 avg-zval = 310.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 39640 bytes = 148672 } seq expand size = 277480 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9816 value = 3.2717 dict = 1.00 all = 3.1954 } Zip/UnZip{ index = 0.1432 value = 0.3056 dict = 1.00 all = 0.3129 } ---------------------------- total value len = 10.836026 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021176 GB avg = 0.015 KB total ukey num = 0.067491893 Billion total entry num = 0.054299622 Billion write speed all = 0.328204578 MB/sec (with seq num) write speed all = 0.316628188 MB/sec (without seq num) 2024/11/23-19:20:36.656082 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276910: 129430 keys, 42003626 bytes, temperature: kWarm 2024/11/23-19:20:36.656148 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360836656114, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276910, "file_size": 42003626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211710735, "table_properties": {"data_size": 40232624, "index_size": 222464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629879, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360834, "oldest_key_time": 0, "file_creation_time": 1732360834, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276910, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:36.693001 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst) /dev/shm/ToplingTemp/Topling-163666-VzW6AS 2024/11/23-19:20:37.348678 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst): first pass time = 0.66's, 204.713'MB/sec 2024/11/23-19:20:38.209554 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst): old prealloc_size = 147639500, real_size = 41062176 2024/11/23-19:20:38.209675 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.29 2024/11/23-19:20:38.209822 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:38.372707 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276911.sst): second pass time = 0.86's, 153.216'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.796'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2263.965'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 972.099'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42015.179'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 153.216'MB/sec zip pipeline throughput = 153.216'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.29 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.33 avg-zval = 315.29 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 117231 bytes = 439632 } seq expand size = 820617 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0343 value = 3.2255 dict = 1.00 all = 3.1308 } Zip/UnZip{ index = 0.1422 value = 0.3100 dict = 1.00 all = 0.3194 } ---------------------------- total value len = 10.968476 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621324 Billion total entry num = 0.054429053 Billion write speed all = 0.331787632 MB/sec (with seq num) write speed all = 0.320184194 MB/sec (without seq num) 2024/11/23-19:20:38.424373 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276911: 129431 keys, 42869931 bytes, temperature: kWarm 2024/11/23-19:20:38.424447 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360838424399, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276911, "file_size": 42869931, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211691538, "table_properties": {"data_size": 40808848, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629508, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360836, "oldest_key_time": 0, "file_creation_time": 1732360836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276911, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:38.460601 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst) /dev/shm/ToplingTemp/Topling-163666-uZJorT 2024/11/23-19:20:39.197615 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst): first pass time = 0.74's, 182.119'MB/sec 2024/11/23-19:20:40.086806 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst): old prealloc_size = 147639500, real_size = 40621024 2024/11/23-19:20:40.086925 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.00 avg-zval = 311.85 2024/11/23-19:20:40.087063 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:40.249273 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276912.sst): second pass time = 0.89's, 148.161'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2371.290'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 973.973'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 33484.624'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 148.161'MB/sec zip pipeline throughput = 148.161'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 311.85 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1021.25 avg-zval = 311.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 78525 bytes = 294480 } seq expand size = 549675 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.8689 value = 3.2612 dict = 1.00 all = 3.1742 } Zip/UnZip{ index = 0.1456 value = 0.3066 dict = 1.00 all = 0.3150 } ---------------------------- total value len = 11.100655 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750753 Billion total entry num = 0.054558482 Billion write speed all = 0.335362165 MB/sec (with seq num) write speed all = 0.323731717 MB/sec (without seq num) 2024/11/23-19:20:40.301606 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276912: 129429 keys, 42283646 bytes, temperature: kWarm 2024/11/23-19:20:40.301673 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360840301640, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276912, "file_size": 42283646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211712824, "table_properties": {"data_size": 40362384, "index_size": 226112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629572, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360838, "oldest_key_time": 0, "file_creation_time": 1732360838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276912, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:40.338566 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst) /dev/shm/ToplingTemp/Topling-163666-yKgoRR 2024/11/23-19:20:41.074963 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst): first pass time = 0.74's, 182.271'MB/sec 2024/11/23-19:20:41.924174 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst): old prealloc_size = 147639500, real_size = 41036080 2024/11/23-19:20:41.924329 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.09 2024/11/23-19:20:41.924485 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:42.090263 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276913.sst): second pass time = 0.85's, 155.137'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 258.090'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2154.676'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 917.449'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44177.132'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.137'MB/sec zip pipeline throughput = 155.137'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 315.09 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.58 avg-zval = 315.09 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 121111 bytes = 454176 } seq expand size = 847777 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9673 value = 3.2277 dict = 1.00 all = 3.1317 } Zip/UnZip{ index = 0.1435 value = 0.3098 dict = 1.00 all = 0.3193 } ---------------------------- total value len = 11.233133 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880178 Billion total entry num = 0.054687907 Billion write speed all = 0.338944598 MB/sec (with seq num) write speed all = 0.327287132 MB/sec (without seq num) 2024/11/23-19:20:42.141535 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276913: 129425 keys, 42858321 bytes, temperature: kWarm 2024/11/23-19:20:42.141605 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360842141569, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276913, "file_size": 42858321, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211693163, "table_properties": {"data_size": 40780640, "index_size": 222912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629398, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360840, "oldest_key_time": 0, "file_creation_time": 1732360840, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276913, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:42.178038 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst) /dev/shm/ToplingTemp/Topling-163666-tVAhAS 2024/11/23-19:20:42.716256 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst): first pass time = 0.54's, 249.392'MB/sec 2024/11/23-19:20:43.642132 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst): old prealloc_size = 147639500, real_size = 40534400 2024/11/23-19:20:43.642269 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.19 2024/11/23-19:20:43.642381 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:43.805286 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276914.sst): second pass time = 0.93's, 142.277'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.319'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1956.647'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 966.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41981.047'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 142.277'MB/sec zip pipeline throughput = 150.907'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.02 avg-zval = 311.19 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1020.45 avg-zval = 311.19 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 63499 bytes = 238144 } seq expand size = 444493 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.8825 value = 3.2682 dict = 1.00 all = 3.1850 } Zip/UnZip{ index = 0.1453 value = 0.3060 dict = 1.00 all = 0.3140 } ---------------------------- total value len = 11.365206 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009605 Billion total entry num = 0.054817334 Billion write speed all = 0.342517081 MB/sec (with seq num) write speed all = 0.330832559 MB/sec (without seq num) 2024/11/23-19:20:43.860873 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276914: 129427 keys, 42139973 bytes, temperature: kWarm 2024/11/23-19:20:43.860941 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360843860907, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276914, "file_size": 42139973, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211713662, "table_properties": {"data_size": 40276208, "index_size": 225664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629215, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360842, "oldest_key_time": 0, "file_creation_time": 1732360842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276914, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:43.899512 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst) /dev/shm/ToplingTemp/Topling-163666-h1epbQ 2024/11/23-19:20:44.577957 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst): first pass time = 0.68's, 197.842'MB/sec 2024/11/23-19:20:45.391491 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst): old prealloc_size = 147639500, real_size = 40633792 2024/11/23-19:20:45.391650 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.97 2024/11/23-19:20:45.391765 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:45.571528 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276915.sst): second pass time = 0.81's, 161.944'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 234.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1651.393'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.245'MB/sec write SST data time = 0.05's, 882.327'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48487.911'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.944'MB/sec zip pipeline throughput = 161.944'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 311.97 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1019.48 avg-zval = 311.97 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 45623 bytes = 171104 } seq expand size = 319361 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9495 value = 3.2600 dict = 1.00 all = 3.1826 } Zip/UnZip{ index = 0.1439 value = 0.3068 dict = 1.00 all = 0.3142 } ---------------------------- total value len = 11.497155 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139033 Billion total entry num = 0.054946762 Billion write speed all = 0.346085440 MB/sec (with seq num) write speed all = 0.334373882 MB/sec (without seq num) 2024/11/23-19:20:45.623507 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276915: 129428 keys, 42172407 bytes, temperature: kWarm 2024/11/23-19:20:45.623579 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360845623544, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276915, "file_size": 42172407, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211708431, "table_properties": {"data_size": 40377776, "index_size": 223488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629541, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360843, "oldest_key_time": 0, "file_creation_time": 1732360843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276915, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:45.666149 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst) /dev/shm/ToplingTemp/Topling-163666-33I9vQ 2024/11/23-19:20:46.389247 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst): first pass time = 0.72's, 185.625'MB/sec 2024/11/23-19:20:47.356300 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst): old prealloc_size = 147639500, real_size = 40478704 2024/11/23-19:20:47.356408 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.77 2024/11/23-19:20:47.356489 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:47.518784 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276916.sst): second pass time = 0.97's, 136.206'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2465.053'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 971.131'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36327.052'MB/sec zip my value time = 0.97's, unzip length = 0.132'GB zip my value throughput = 136.206'MB/sec zip pipeline throughput = 136.206'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.77 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.05 avg-zval = 310.77 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 38127 bytes = 142992 } seq expand size = 266889 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9737 value = 3.2725 dict = 1.00 all = 3.1965 } Zip/UnZip{ index = 0.1434 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.629052 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268464 Billion total entry num = 0.055076193 Billion write speed all = 0.349650382 MB/sec (with seq num) write speed all = 0.337911845 MB/sec (without seq num) 2024/11/23-19:20:47.569588 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276916: 129431 keys, 41989712 bytes, temperature: kWarm 2024/11/23-19:20:47.569658 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360847569623, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276916, "file_size": 41989712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211671063, "table_properties": {"data_size": 40223456, "index_size": 222720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629598, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360845, "oldest_key_time": 0, "file_creation_time": 1732360845, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276916, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:47.611353 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst) /dev/shm/ToplingTemp/Topling-163666-m6CpcT 2024/11/23-19:20:48.340599 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst): first pass time = 0.73's, 184.060'MB/sec 2024/11/23-19:20:49.168499 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst): old prealloc_size = 147639500, real_size = 40473584 2024/11/23-19:20:49.168619 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.74 2024/11/23-19:20:49.168715 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:20:49.333127 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276917.sst): second pass time = 0.83's, 159.302'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 254.963'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2284.634'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 913.804'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42312.045'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.302'MB/sec zip pipeline throughput = 159.302'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.74 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.85 avg-zval = 310.74 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 34050 bytes = 127696 } seq expand size = 238350 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9876 value = 3.2728 dict = 1.00 all = 3.1980 } Zip/UnZip{ index = 0.1431 value = 0.3055 dict = 1.00 all = 0.3127 } ---------------------------- total value len = 11.760920 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032048 GB avg = 0.015 KB total ukey num = 0.068397893 Billion total entry num = 0.055205622 Billion write speed all = 0.353215450 MB/sec (with seq num) write speed all = 0.341449896 MB/sec (without seq num) 2024/11/23-19:20:49.385132 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276917: 129429 keys, 41969036 bytes, temperature: kWarm 2024/11/23-19:20:49.385202 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360849385166, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276917, "file_size": 41969036, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211712271, "table_properties": {"data_size": 40218784, "index_size": 222272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629819, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360847, "oldest_key_time": 0, "file_creation_time": 1732360847, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276917, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:49.425423 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst) /dev/shm/ToplingTemp/Topling-163666-dd7e1Q 2024/11/23-19:20:49.459133 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst): first pass time = 0.03's, 161.034'MB/sec 2024/11/23-19:20:49.501665 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst): old prealloc_size = 147639500, real_size = 1704992 2024/11/23-19:20:49.501721 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 5229 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 2.00 raw-val = 0.0053 GB zip-val = 0.0017 GB avg-val =1016.96 avg-zval = 323.79 2024/11/23-19:20:49.501765 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.005 GB 2024/11/23-19:20:49.510628 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00/276918.sst): second pass time = 0.04's, 127.090'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 197.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 291.666'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.193'MB/sec write SST data time = 0.00's, 817.225'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.053'MB, throughput = 1604.081'MB/sec zip my value time = 0.04's, unzip length = 0.005'GB zip my value throughput = 127.090'MB/sec zip pipeline throughput = 127.090'MB/sec entries = 5229 avg-key = 20.00 avg-zkey = 2.00 avg-val = 1016.96 avg-zval = 323.79 usrkeys = 5229 avg-key = 12.00 avg-zkey = 2.00 avg-val = 1019.68 avg-zval = 323.79 TagRS{ kinds = RS_Key0_TagN bytes = 920 } TagArray{ size = 2032 bytes = 7632 } seq expand size = 14224 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0001 GB value = 0.0053 GB dict = 0.05 MB all = 0.0054 GB } __ZipSize{ index = 0.0000 GB value = 0.0017 GB dict = 0.05 MB all = 0.0018 GB } UnZip/Zip{ index = 6.0150 value = 3.1408 dict = 1.00 all = 3.0701 } Zip/UnZip{ index = 0.1663 value = 0.3184 dict = 1.00 all = 0.3257 } ---------------------------- total value len = 11.766252 GB avg = 0.213 KB (by entry num) total key len = 1.497875 GB avg = 0.027 KB total ukey len = 1.032110 GB avg = 0.015 KB total ukey num = 0.068403122 Billion total entry num = 0.055210851 Billion write speed all = 0.353358608 MB/sec (with seq num) write speed all = 0.341591996 MB/sec (without seq num) 2024/11/23-19:20:49.521483 139957475337792 [compaction_job.cc:1948] [default] [JOB 34047] Generated table #276918: 5229 keys, 1766145 bytes, temperature: kWarm 2024/11/23-19:20:49.521535 139957475337792 EVENT_LOG_v1 {"time_micros": 1732360849521506, "cf_name": "default", "job": 34047, "event": "table_file_creation", "file_number": 276918, "file_size": 1766145, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4211714408, "table_properties": {"data_size": 1693072, "index_size": 10432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 104580, "raw_average_key_size": 20, "raw_value_size": 5317685, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 5229, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732360849, "oldest_key_time": 0, "file_creation_time": 1732360849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 276918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:20:49.531656 139957475337792 [compaction_job.cc:703] job-34047: subcompact[0], size: 0.936256 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34047/att-00 [276896,276897,276898,276899,276900,276901,276902,276903,276904,276905,276906,276907,276908,276909,276910,276911,276912,276913,276914,276915,276916,276917,276918] 2024/11/23-19:20:49.669515 139957475337792 (Original Log Time 2024/11/23-19:20:05.423526) EVENT_LOG_v1 {"time_micros": 1732360805423486, "job": 34047, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [276892], "files_L3": [276821, 276822, 276823, 276824, 276825, 276826, 276827, 276828, 276829, 276830, 276831, 276832, 276833, 276834, 276835, 276836, 276837, 276838, 276839, 276840, 276841, 276842, 276843], "score": 0.994195, "input_data_size": 968089106, "oldest_snapshot_seqno": 4211731987} 2024/11/23-19:20:49.669518 139957475337792 (Original Log Time 2024/11/23-19:20:49.667803) [compaction_job.cc:2022] [default] [JOB 34047] Compacted 1@2 + 23@3 files to L3 => 936255859 bytes 2024/11/23-19:20:49.669520 139957475337792 (Original Log Time 2024/11/23-19:20:49.669442) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 21.9 rd, 21.2 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(31.2, 892.1 +0.0 blob) out(892.9 +0.0 blob), read-write-amplify(58.2) write-amplify(28.6) OK, records in: 2881983, records dropped: 29296 output_compression: Snappy 2024/11/23-19:20:49.669523 139957475337792 (Original Log Time 2024/11/23-19:20:49.669487) EVENT_LOG_v1 {"time_micros": 1732360849669467, "cf": "default", "job": 34047, "event": "compaction_finished", "compaction_time_micros": 44108137, "compaction_time_cpu_micros": 8372138, "output_level": 3, "num_output_files": 23, "total_output_size": 936255859, "num_input_records": 2881983, "num_output_records": 2852687, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 701691617, "file_range_sync_nanos": 0, "file_fsync_nanos": 17489, "file_prepare_write_nanos": 57409, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-19:20:49.814597 139957475337792 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732360805,"executesMs":44380,"compactionJobId":34047,"attempt":0,"compactionInputRawBytes":2988601800,"compactionInputZipBytes":968089106,"compactionOutputRawBytes":0,"compactionOutputZipBytes":936255859,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4c801fc57a048c33f1bfe8ea9a353f2a-95da309c7170a648-00"}}, response = 2024/11/23-19:20:49.814691 139957475337792 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34047/att-00: olev 3, work 44.381 s, result 6.004 ms, install 1.735 ms, input{raw 2.783 GiB zip 923.242 MiB}