2024/11/23-17:09:56.081141 139957299091008 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00: fork to child time = 0.007277 sec 2024/11/23-17:09:56.082075 139957299091008 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00 2024/11/23-17:09:56.090718 139957299091008 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00 2024/11/23-17:09:56.091973 139957299091008 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:09:56.092305 139957299091008 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:09:56.162664 139957299091008 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31565/att-00/MANIFEST-206347 2024/11/23-17:09:56.179018 139957299091008 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31565/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 271478, last_sequence is 4136360062, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271328 2024/11/23-17:09:56.179026 139957299091008 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:09:56.179185 139957299091008 [version_set.cc:5721] Creating manifest 271478 2024/11/23-17:09:56.214724 139957299091008 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:09:56.239191 139957299091008 [compaction_job.cc:2437] [default] [JOB 31565] Compacting 1@2 + 22@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:09:56.239216 139957299091008 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23617 Base level 2, inputs: [271455(27MB)], [271376(36MB) 271377(36MB) 271406(37MB) 271407(39MB) 271408(34MB) 271409(36MB) 271410(35MB) 271411(35MB) 271412(35MB) 271413(36MB) 271414(36MB) 271415(38MB) 271416(35MB) 271417(34MB) 271418(37MB) 271419(34MB) 271420(37MB) 271421(35MB) 271422(34MB) 271423(36MB) 271424(36MB) 271474(37MB)] 2024/11/23-17:09:56.248353 139957299091008 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:09:56.248387 139957299091008 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:09:56.248411 139957299091008 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:09:56.248419 139957299091008 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:09:56.248435 139957299091008 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:09:56.830322 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst) /dev/shm/ToplingTemp/Topling-127513-ed05Ea 2024/11/23-17:09:57.383826 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst): first pass time = 0.55's, 242.506'MB/sec 2024/11/23-17:09:58.899870 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst): old prealloc_size = 147639500, real_size = 34736144 2024/11/23-17:09:58.900592 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925614 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 117.00 avg-zval = 35.38 2024/11/23-17:09:58.901425 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:09:59.895034 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271479.sst): second pass time = 1.52's, 71.481'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 38.788'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2511.324'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.15's, 253.011'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28951.057'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.481'MB/sec zip pipeline throughput = 71.481'MB/sec entries = 925614 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 35.38 usrkeys = 925614 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 35.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925614 bytes = 3008256 } seq expand size = 6479298 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5141 value = 3.3075 dict = 1.00 all = 3.4772 } Zip/UnZip{ index = 0.0951 value = 0.3023 dict = 1.00 all = 0.2876 } ---------------------------- total value len = 8.963400 GB avg = 0.168 KB (by entry num) total key len = 1.466738 GB avg = 0.028 KB total ukey len = 1.016390 GB avg = 0.015 KB total ukey num = 0.066476049 Billion total entry num = 0.053283778 Billion write speed all = 0.351328827 MB/sec (with seq num) write speed all = 0.336970339 MB/sec (without seq num) 2024/11/23-17:09:59.945267 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271479: 925614 keys, 38599578 bytes, temperature: kWarm 2024/11/23-17:09:59.945362 139957299091008 EVENT_LOG_v1 {"time_micros": 1732352999945306, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271479, "file_size": 38599578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024658, "largest_seqno": 4136223825, "table_properties": {"data_size": 32743872, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917192, "raw_average_key_size": 28, "raw_value_size": 108300572, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925614, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352996, "oldest_key_time": 0, "file_creation_time": 1732352996, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:09:59.983351 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst) /dev/shm/ToplingTemp/Topling-127513-BnKOo9 2024/11/23-17:10:01.296190 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst): first pass time = 1.31's, 102.238'MB/sec 2024/11/23-17:10:03.016566 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst): old prealloc_size = 147639500, real_size = 34920304 2024/11/23-17:10:03.017205 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924569 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 117.17 avg-zval = 35.62 2024/11/23-17:10:03.017959 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:04.052283 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271480.sst): second pass time = 1.71's, 63.190'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2862.117'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 305.232'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31794.196'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.190'MB/sec zip pipeline throughput = 76.609'MB/sec entries = 924569 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.17 avg-zval = 35.62 usrkeys = 924569 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.17 avg-zval = 35.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924569 bytes = 3004864 } seq expand size = 6471983 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5133 value = 3.2897 dict = 1.00 all = 3.4609 } Zip/UnZip{ index = 0.0951 value = 0.3040 dict = 1.00 all = 0.2889 } ---------------------------- total value len = 9.078202 GB avg = 0.167 KB (by entry num) total key len = 1.492626 GB avg = 0.028 KB total ukey len = 1.034882 GB avg = 0.015 KB total ukey num = 0.067400618 Billion total entry num = 0.054208347 Billion write speed all = 0.356017970 MB/sec (with seq num) write speed all = 0.341412381 MB/sec (without seq num) 2024/11/23-17:10:04.102739 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271480: 924569 keys, 38781336 bytes, temperature: kWarm 2024/11/23-17:10:04.102814 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353004102776, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271480, "file_size": 38781336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4087041805, "largest_seqno": 4136224137, "table_properties": {"data_size": 32930144, "index_size": 1758848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25887932, "raw_average_key_size": 28, "raw_value_size": 108329910, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924569, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732352999, "oldest_key_time": 0, "file_creation_time": 1732352999, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:04.141264 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst) /dev/shm/ToplingTemp/Topling-127513-DrvGK8 2024/11/23-17:10:05.205344 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst): first pass time = 1.06's, 126.140'MB/sec 2024/11/23-17:10:06.920045 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst): old prealloc_size = 147639500, real_size = 35880320 2024/11/23-17:10:06.920602 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927711 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.68 avg-zval = 36.52 2024/11/23-17:10:06.921345 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:07.900158 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271481.sst): second pass time = 1.71's, 63.155'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 40.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3301.456'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.13's, 314.603'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34832.146'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.155'MB/sec zip pipeline throughput = 85.268'MB/sec entries = 927711 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 36.52 usrkeys = 927711 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 36.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927711 bytes = 3131040 } seq expand size = 6493977 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.5165 value = 3.1945 dict = 1.00 all = 3.3667 } Zip/UnZip{ index = 0.0951 value = 0.3130 dict = 1.00 all = 0.2970 } ---------------------------- total value len = 9.192937 GB avg = 0.167 KB (by entry num) total key len = 1.518602 GB avg = 0.028 KB total ukey len = 1.053436 GB avg = 0.015 KB total ukey num = 0.068328329 Billion total entry num = 0.055136058 Billion write speed all = 0.360710297 MB/sec (with seq num) write speed all = 0.345856676 MB/sec (without seq num) 2024/11/23-17:10:07.951566 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271481: 927711 keys, 39865824 bytes, temperature: kWarm 2024/11/23-17:10:07.951636 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353007951599, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271481, "file_size": 39865824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4136223971, "table_properties": {"data_size": 33883952, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975908, "raw_average_key_size": 28, "raw_value_size": 108241894, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927711, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353004, "oldest_key_time": 0, "file_creation_time": 1732353004, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:07.988792 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst) /dev/shm/ToplingTemp/Topling-127513-AN9bFa 2024/11/23-17:10:09.074374 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst): first pass time = 1.09's, 123.641'MB/sec 2024/11/23-17:10:11.008044 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst): old prealloc_size = 147639500, real_size = 36726864 2024/11/23-17:10:11.008599 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933978 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0347 GB avg-val = 115.71 avg-zval = 37.16 2024/11/23-17:10:11.009482 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:12.298802 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271482.sst): second pass time = 1.93's, 55.907'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 31.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3355.086'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.217'MB/sec write SST data time = 0.22's, 190.369'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33731.673'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 55.907'MB/sec zip pipeline throughput = 124.005'MB/sec entries = 933978 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.71 avg-zval = 37.16 usrkeys = 933978 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.71 avg-zval = 37.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933978 bytes = 3502432 } seq expand size = 6537846 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4459 value = 3.1139 dict = 1.00 all = 3.2670 } Zip/UnZip{ index = 0.0957 value = 0.3211 dict = 1.00 all = 0.3061 } ---------------------------- total value len = 9.307542 GB avg = 0.166 KB (by entry num) total key len = 1.544753 GB avg = 0.028 KB total ukey len = 1.072115 GB avg = 0.015 KB total ukey num = 0.069262307 Billion total entry num = 0.056070036 Billion write speed all = 0.365396108 MB/sec (with seq num) write speed all = 0.350293110 MB/sec (without seq num) 2024/11/23-17:10:12.349821 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271482: 933978 keys, 41083490 bytes, temperature: kWarm 2024/11/23-17:10:12.349886 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353012349850, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271482, "file_size": 41083490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4136223324, "table_properties": {"data_size": 34704960, "index_size": 1788224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26151384, "raw_average_key_size": 28, "raw_value_size": 108066380, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933978, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353007, "oldest_key_time": 0, "file_creation_time": 1732353007, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:12.387867 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst) /dev/shm/ToplingTemp/Topling-127513-SLuKv8 2024/11/23-17:10:13.322121 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst): first pass time = 0.93's, 143.668'MB/sec 2024/11/23-17:10:14.984866 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst): old prealloc_size = 147639500, real_size = 35186688 2024/11/23-17:10:14.985546 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1083224 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.91 avg-zval = 30.32 2024/11/23-17:10:14.985768 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:16.019572 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271483.sst): second pass time = 1.66's, 62.509'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 35.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.967'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.333'MB/sec write SST data time = 0.22's, 164.906'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 38172.628'MB/sec zip my value time = 1.66's, unzip length = 0.104'GB zip my value throughput = 62.509'MB/sec zip pipeline throughput = 62.509'MB/sec entries = 1083224 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.91 avg-zval = 30.32 usrkeys = 1083224 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.48 avg-zval = 30.32 TagRS{ kinds = RS_Key0_TagN bytes = 169360 } TagArray{ size = 88113 bytes = 275376 } seq expand size = 616791 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4478 value = 3.1632 dict = 1.00 all = 3.6868 } Zip/UnZip{ index = 0.0957 value = 0.3161 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 9.412046 GB avg = 0.165 KB (by entry num) total key len = 1.575084 GB avg = 0.028 KB total ukey len = 1.093780 GB avg = 0.016 KB total ukey num = 0.070345531 Billion total entry num = 0.057153260 Billion write speed all = 0.369889639 MB/sec (with seq num) write speed all = 0.354496794 MB/sec (without seq num) 2024/11/23-17:10:16.065178 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271483: 1083224 keys, 36404639 bytes, temperature: kWarm 2024/11/23-17:10:16.065237 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353016065205, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271483, "file_size": 36404639, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32842096, "index_size": 2073600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30330272, "raw_average_key_size": 28, "raw_value_size": 103887473, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1083224, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353012, "oldest_key_time": 0, "file_creation_time": 1732353012, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:16.102180 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst) /dev/shm/ToplingTemp/Topling-127513-mEuLu9 2024/11/23-17:10:16.966241 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst): first pass time = 0.86's, 155.342'MB/sec 2024/11/23-17:10:18.748724 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst): old prealloc_size = 147639500, real_size = 35254512 2024/11/23-17:10:18.749408 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922611 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.48 avg-zval = 36.06 2024/11/23-17:10:18.750016 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:19.849770 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271484.sst): second pass time = 1.78's, 60.848'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 34.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2661.830'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.246'MB/sec write SST data time = 0.16's, 233.088'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35787.185'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.848'MB/sec zip pipeline throughput = 60.848'MB/sec entries = 922611 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 36.06 usrkeys = 922611 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.49 avg-zval = 36.06 TagRS{ kinds = RS_Key0_TagN bytes = 144240 } TagArray{ size = 529319 bytes = 1984960 } seq expand size = 3705233 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5133 value = 3.2579 dict = 1.00 all = 3.5094 } Zip/UnZip{ index = 0.0951 value = 0.3069 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 9.524136 GB avg = 0.164 KB (by entry num) total key len = 1.600917 GB avg = 0.028 KB total ukey len = 1.112232 GB avg = 0.016 KB total ukey num = 0.071268142 Billion total entry num = 0.058075871 Billion write speed all = 0.374484630 MB/sec (with seq num) write speed all = 0.358845318 MB/sec (without seq num) 2024/11/23-17:10:19.899138 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271484: 922611 keys, 38245660 bytes, temperature: kWarm 2024/11/23-17:10:19.899207 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353019899172, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271484, "file_size": 38245660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3989178139, "table_properties": {"data_size": 33268512, "index_size": 1755136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833108, "raw_average_key_size": 28, "raw_value_size": 108384722, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353016, "oldest_key_time": 0, "file_creation_time": 1732353016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:19.941883 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst) /dev/shm/ToplingTemp/Topling-127513-gzsux7 2024/11/23-17:10:21.067205 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst): first pass time = 1.13's, 119.274'MB/sec 2024/11/23-17:10:22.445948 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst): old prealloc_size = 147639500, real_size = 32727776 2024/11/23-17:10:22.446679 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.10 avg-zval = 32.98 2024/11/23-17:10:22.447567 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:23.469928 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271485.sst): second pass time = 1.38's, 78.486'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.090'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2554.449'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.16's, 232.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34758.487'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.486'MB/sec zip pipeline throughput = 78.486'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 32.98 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5148 value = 3.5198 dict = 1.00 all = 3.6319 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 9.638794 GB avg = 0.163 KB (by entry num) total key len = 1.626997 GB avg = 0.028 KB total ukey len = 1.130861 GB avg = 0.016 KB total ukey num = 0.072199571 Billion total entry num = 0.059007300 Billion write speed all = 0.379175852 MB/sec (with seq num) write speed all = 0.363287651 MB/sec (without seq num) 2024/11/23-17:10:23.517566 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271485: 931429 keys, 36955304 bytes, temperature: kWarm 2024/11/23-17:10:23.517637 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353023517601, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271485, "file_size": 36955304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30723088, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353019, "oldest_key_time": 0, "file_creation_time": 1732353019, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271485, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:23.554040 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst) /dev/shm/ToplingTemp/Topling-127513-uA93n9 2024/11/23-17:10:24.925410 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst): first pass time = 1.37's, 97.874'MB/sec 2024/11/23-17:10:26.449476 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst): old prealloc_size = 147639500, real_size = 32746080 2024/11/23-17:10:26.450033 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.12 avg-zval = 33.00 2024/11/23-17:10:26.450825 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:27.329867 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271486.sst): second pass time = 1.52's, 70.987'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.841'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 293.310'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 39183.258'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 70.987'MB/sec zip pipeline throughput = 70.987'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.12 avg-zval = 33.00 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.12 avg-zval = 33.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4426 value = 3.5191 dict = 1.00 all = 3.6416 } Zip/UnZip{ index = 0.0958 value = 0.2842 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.753454 GB avg = 0.163 KB (by entry num) total key len = 1.653074 GB avg = 0.028 KB total ukey len = 1.149487 GB avg = 0.016 KB total ukey num = 0.073130880 Billion total entry num = 0.059938609 Billion write speed all = 0.383862805 MB/sec (with seq num) write speed all = 0.367725938 MB/sec (without seq num) 2024/11/23-17:10:27.375234 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271486: 931309 keys, 36857120 bytes, temperature: kWarm 2024/11/23-17:10:27.375314 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353027375277, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271486, "file_size": 36857120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30729360, "index_size": 1783680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353023, "oldest_key_time": 0, "file_creation_time": 1732353023, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271486, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:27.410967 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst) /dev/shm/ToplingTemp/Topling-127513-D8jpsa 2024/11/23-17:10:28.470320 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst): first pass time = 1.06's, 126.704'MB/sec 2024/11/23-17:10:30.110436 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst): old prealloc_size = 147639500, real_size = 32910944 2024/11/23-17:10:30.111111 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.09 avg-zval = 33.18 2024/11/23-17:10:30.112083 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:31.039798 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271487.sst): second pass time = 1.64's, 65.963'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.088'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2770.899'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.18's, 208.355'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32944.506'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.963'MB/sec zip pipeline throughput = 97.055'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.18 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5133 value = 3.4989 dict = 1.00 all = 3.6027 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 9.868111 GB avg = 0.162 KB (by entry num) total key len = 1.679154 GB avg = 0.028 KB total ukey len = 1.168116 GB avg = 0.016 KB total ukey num = 0.074062342 Billion total entry num = 0.060870071 Billion write speed all = 0.388550539 MB/sec (with seq num) write speed all = 0.372164946 MB/sec (without seq num) 2024/11/23-17:10:31.087949 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271487: 931462 keys, 37254988 bytes, temperature: kWarm 2024/11/23-17:10:31.088056 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353031088007, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271487, "file_size": 37254988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30905936, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353027, "oldest_key_time": 0, "file_creation_time": 1732353027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271487, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:31.164296 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst) /dev/shm/ToplingTemp/Topling-127513-vGKLr7 2024/11/23-17:10:32.234991 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst): first pass time = 1.07's, 125.364'MB/sec 2024/11/23-17:10:33.953563 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst): old prealloc_size = 147639500, real_size = 34506368 2024/11/23-17:10:33.954165 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.04 avg-zval = 35.12 2024/11/23-17:10:33.955008 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:34.992899 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271488.sst): second pass time = 1.72's, 63.051'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3040.111'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 293.853'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35463.381'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 63.051'MB/sec zip pipeline throughput = 113.355'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.04 avg-zval = 35.12 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.04 avg-zval = 35.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4446 value = 3.3322 dict = 1.00 all = 3.4667 } Zip/UnZip{ index = 0.0957 value = 0.3001 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 9.982895 GB avg = 0.162 KB (by entry num) total key len = 1.705066 GB avg = 0.028 KB total ukey len = 1.186624 GB avg = 0.016 KB total ukey num = 0.074987747 Billion total entry num = 0.061795476 Billion write speed all = 0.393232457 MB/sec (with seq num) write speed all = 0.376599967 MB/sec (without seq num) 2024/11/23-17:10:35.040947 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271488: 925405 keys, 38716254 bytes, temperature: kWarm 2024/11/23-17:10:35.041036 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353035040995, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271488, "file_size": 38716254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32502832, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353031, "oldest_key_time": 0, "file_creation_time": 1732353031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271488, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:35.101196 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst) /dev/shm/ToplingTemp/Topling-127513-2uDBea 2024/11/23-17:10:36.028609 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst): first pass time = 0.93's, 144.731'MB/sec 2024/11/23-17:10:37.562809 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst): old prealloc_size = 147639500, real_size = 35208688 2024/11/23-17:10:37.563394 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925306 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.05 avg-zval = 35.89 2024/11/23-17:10:37.564189 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:38.432101 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271489.sst): second pass time = 1.53's, 70.636'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 45.346'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3149.509'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 297.350'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35335.383'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.636'MB/sec zip pipeline throughput = 70.636'MB/sec entries = 925306 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.05 avg-zval = 35.89 usrkeys = 925306 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.05 avg-zval = 35.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925306 bytes = 3354256 } seq expand size = 6477142 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4453 value = 3.2618 dict = 1.00 all = 3.4050 } Zip/UnZip{ index = 0.0957 value = 0.3066 dict = 1.00 all = 0.2937 } ---------------------------- total value len = 10.097682 GB avg = 0.161 KB (by entry num) total key len = 1.730974 GB avg = 0.028 KB total ukey len = 1.205130 GB avg = 0.016 KB total ukey num = 0.075913053 Billion total entry num = 0.062720782 Billion write speed all = 0.397919985 MB/sec (with seq num) write speed all = 0.381040399 MB/sec (without seq num) 2024/11/23-17:10:38.480836 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271489: 925306 keys, 39418044 bytes, temperature: kWarm 2024/11/23-17:10:38.480904 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353038480868, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271489, "file_size": 39418044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 33205472, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908568, "raw_average_key_size": 28, "raw_value_size": 108309192, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925306, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353035, "oldest_key_time": 0, "file_creation_time": 1732353035, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271489, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:38.520216 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst) /dev/shm/ToplingTemp/Topling-127513-uTGlq9 2024/11/23-17:10:39.573087 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst): first pass time = 1.05's, 127.483'MB/sec 2024/11/23-17:10:40.911554 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst): old prealloc_size = 147639500, real_size = 35853440 2024/11/23-17:10:40.912105 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929590 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.38 avg-zval = 36.40 2024/11/23-17:10:40.912817 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:41.714183 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271490.sst): second pass time = 1.34's, 80.882'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 49.408'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.844'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.345'MB/sec write SST data time = 0.13's, 310.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35310.665'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.882'MB/sec zip pipeline throughput = 80.882'MB/sec entries = 929590 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.38 avg-zval = 36.40 usrkeys = 929590 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.01 avg-zval = 36.40 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 746593 bytes = 2799744 } seq expand size = 5226151 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.4465 value = 3.1970 dict = 1.00 all = 3.3847 } Zip/UnZip{ index = 0.0957 value = 0.3128 dict = 1.00 all = 0.2955 } ---------------------------- total value len = 10.211097 GB avg = 0.160 KB (by entry num) total key len = 1.757003 GB avg = 0.028 KB total ukey len = 1.223722 GB avg = 0.016 KB total ukey num = 0.076842643 Billion total entry num = 0.063650372 Billion write speed all = 0.402566482 MB/sec (with seq num) write speed all = 0.385438613 MB/sec (without seq num) 2024/11/23-17:10:41.764093 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271490: 929590 keys, 39654748 bytes, temperature: kWarm 2024/11/23-17:10:41.764166 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353041764128, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271490, "file_size": 39654748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 33841136, "index_size": 1779712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028520, "raw_average_key_size": 28, "raw_value_size": 108189352, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929590, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353038, "oldest_key_time": 0, "file_creation_time": 1732353038, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271490, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:41.800179 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst) /dev/shm/ToplingTemp/Topling-127513-GRDwd7 2024/11/23-17:10:42.824926 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst): first pass time = 1.02's, 130.981'MB/sec 2024/11/23-17:10:44.147711 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst): old prealloc_size = 147639500, real_size = 33075504 2024/11/23-17:10:44.148294 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.49 avg-zval = 33.45 2024/11/23-17:10:44.149069 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:45.012126 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271491.sst): second pass time = 1.32's, 81.853'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.706'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 296.171'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32253.137'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.853'MB/sec zip pipeline throughput = 81.853'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.45 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5141 value = 3.4820 dict = 1.00 all = 3.5987 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 10.325808 GB avg = 0.160 KB (by entry num) total key len = 1.783012 GB avg = 0.028 KB total ukey len = 1.242300 GB avg = 0.016 KB total ukey num = 0.077771548 Billion total entry num = 0.064579277 Billion write speed all = 0.407254656 MB/sec (with seq num) write speed all = 0.389878752 MB/sec (without seq num) 2024/11/23-17:10:45.059339 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271491: 928905 keys, 37296170 bytes, temperature: kWarm 2024/11/23-17:10:45.059414 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353045059377, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271491, "file_size": 37296170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31076128, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353041, "oldest_key_time": 0, "file_creation_time": 1732353041, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271491, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:45.097436 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst) /dev/shm/ToplingTemp/Topling-127513-QuLdT7 2024/11/23-17:10:46.112913 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst): first pass time = 1.02's, 132.179'MB/sec 2024/11/23-17:10:47.418210 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst): old prealloc_size = 147639500, real_size = 32330448 2024/11/23-17:10:47.418812 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935326 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0303 GB avg-val = 115.50 avg-zval = 32.41 2024/11/23-17:10:47.419648 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:48.277517 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271492.sst): second pass time = 1.30's, 82.851'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.560'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3064.815'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 291.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33770.773'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.851'MB/sec zip pipeline throughput = 82.851'MB/sec entries = 935326 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.50 avg-zval = 32.41 usrkeys = 935326 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.50 avg-zval = 32.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935326 bytes = 3390576 } seq expand size = 6547282 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5125 value = 3.5633 dict = 1.00 all = 3.6701 } Zip/UnZip{ index = 0.0951 value = 0.2806 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 10.440384 GB avg = 0.159 KB (by entry num) total key len = 1.809201 GB avg = 0.028 KB total ukey len = 1.261007 GB avg = 0.016 KB total ukey num = 0.078706874 Billion total entry num = 0.065514603 Billion write speed all = 0.411943750 MB/sec (with seq num) write speed all = 0.394318120 MB/sec (without seq num) 2024/11/23-17:10:48.323839 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271492: 935326 keys, 36570900 bytes, temperature: kWarm 2024/11/23-17:10:48.323909 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353048323874, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271492, "file_size": 36570900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30316992, "index_size": 1779456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26189128, "raw_average_key_size": 28, "raw_value_size": 108028628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353045, "oldest_key_time": 0, "file_creation_time": 1732353045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271492, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:48.362894 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst) /dev/shm/ToplingTemp/Topling-127513-E2oct9 2024/11/23-17:10:49.435763 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst): first pass time = 1.07's, 125.106'MB/sec 2024/11/23-17:10:50.790248 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst): old prealloc_size = 147639500, real_size = 35035360 2024/11/23-17:10:50.790827 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930506 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.24 avg-zval = 35.49 2024/11/23-17:10:50.791644 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:51.626660 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271493.sst): second pass time = 1.35's, 79.927'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.079'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3195.243'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 313.482'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32264.268'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.927'MB/sec zip pipeline throughput = 79.927'MB/sec entries = 930506 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.24 avg-zval = 35.49 usrkeys = 930506 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.24 avg-zval = 35.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930506 bytes = 3489408 } seq expand size = 6513542 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4440 value = 3.2756 dict = 1.00 all = 3.4086 } Zip/UnZip{ index = 0.0957 value = 0.3053 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 10.555061 GB avg = 0.159 KB (by entry num) total key len = 1.835256 GB avg = 0.028 KB total ukey len = 1.279617 GB avg = 0.016 KB total ukey num = 0.079637380 Billion total entry num = 0.066445109 Billion write speed all = 0.416629508 MB/sec (with seq num) write speed all = 0.398755555 MB/sec (without seq num) 2024/11/23-17:10:51.675291 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271493: 930506 keys, 39376110 bytes, temperature: kWarm 2024/11/23-17:10:51.675363 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353051675325, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271493, "file_size": 39376110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33020688, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054168, "raw_average_key_size": 28, "raw_value_size": 108163660, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930506, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353048, "oldest_key_time": 0, "file_creation_time": 1732353048, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271493, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:51.712230 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst) /dev/shm/ToplingTemp/Topling-127513-i1GQ67 2024/11/23-17:10:52.669780 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst): first pass time = 0.96's, 140.175'MB/sec 2024/11/23-17:10:53.999051 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst): old prealloc_size = 147639500, real_size = 30392240 2024/11/23-17:10:53.999591 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0284 GB avg-val = 116.13 avg-zval = 30.48 2024/11/23-17:10:54.000388 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:54.818557 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271494.sst): second pass time = 1.33's, 81.458'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.245'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.039'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.12's, 279.228'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28254.579'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.458'MB/sec zip pipeline throughput = 81.458'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 30.48 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 30.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.5123 value = 3.8095 dict = 1.00 all = 3.8770 } Zip/UnZip{ index = 0.0951 value = 0.2625 dict = 1.00 all = 0.2579 } ---------------------------- total value len = 10.669723 GB avg = 0.158 KB (by entry num) total key len = 1.861330 GB avg = 0.028 KB total ukey len = 1.298242 GB avg = 0.016 KB total ukey num = 0.080568622 Billion total entry num = 0.067376351 Billion write speed all = 0.421316610 MB/sec (with seq num) write speed all = 0.403194094 MB/sec (without seq num) 2024/11/23-17:10:54.863112 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271494: 931242 keys, 34619408 bytes, temperature: kWarm 2024/11/23-17:10:54.863182 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353054863146, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271494, "file_size": 34619408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 28387552, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353051, "oldest_key_time": 0, "file_creation_time": 1732353051, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271494, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:54.900377 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst) /dev/shm/ToplingTemp/Topling-127513-PIKD27 2024/11/23-17:10:55.754167 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst): first pass time = 0.85's, 157.209'MB/sec 2024/11/23-17:10:57.101458 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst): old prealloc_size = 147639500, real_size = 35240976 2024/11/23-17:10:57.102037 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.46 avg-zval = 36.04 2024/11/23-17:10:57.102902 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:10:57.945039 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271495.sst): second pass time = 1.35's, 80.491'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.896'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3134.590'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 314.071'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31825.870'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.491'MB/sec zip pipeline throughput = 80.491'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.04 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5135 value = 3.2591 dict = 1.00 all = 3.3928 } Zip/UnZip{ index = 0.0951 value = 0.3068 dict = 1.00 all = 0.2947 } ---------------------------- total value len = 10.784564 GB avg = 0.158 KB (by entry num) total key len = 1.887166 GB avg = 0.028 KB total ukey len = 1.316696 GB avg = 0.016 KB total ukey num = 0.081491319 Billion total entry num = 0.068299048 Billion write speed all = 0.426001633 MB/sec (with seq num) write speed all = 0.407632867 MB/sec (without seq num) 2024/11/23-17:10:57.993118 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271495: 922697 keys, 39559338 bytes, temperature: kWarm 2024/11/23-17:10:57.993190 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353057993155, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271495, "file_size": 39559338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33254848, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353054, "oldest_key_time": 0, "file_creation_time": 1732353054, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271495, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:10:58.031188 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst) /dev/shm/ToplingTemp/Topling-127513-yYW5aa 2024/11/23-17:10:59.061960 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst): first pass time = 1.03's, 130.217'MB/sec 2024/11/23-17:11:00.383782 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst): old prealloc_size = 147639500, real_size = 32286960 2024/11/23-17:11:00.384330 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.15 avg-zval = 32.51 2024/11/23-17:11:00.385168 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:11:01.231823 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271496.sst): second pass time = 1.32's, 81.869'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.195'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3402.274'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.345'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31418.167'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.869'MB/sec zip pipeline throughput = 81.869'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.51 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4438 value = 3.5727 dict = 1.00 all = 3.6642 } Zip/UnZip{ index = 0.0958 value = 0.2799 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 10.899229 GB avg = 0.157 KB (by entry num) total key len = 1.913236 GB avg = 0.028 KB total ukey len = 1.335317 GB avg = 0.016 KB total ukey num = 0.082422402 Billion total entry num = 0.069230131 Billion write speed all = 0.430685326 MB/sec (with seq num) write speed all = 0.412068205 MB/sec (without seq num) 2024/11/23-17:11:01.276245 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271496: 931083 keys, 36629836 bytes, temperature: kWarm 2024/11/23-17:11:01.276317 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353061276281, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271496, "file_size": 36629836, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30270944, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353058, "oldest_key_time": 0, "file_creation_time": 1732353058, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271496, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:01.325377 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst) /dev/shm/ToplingTemp/Topling-127513-LDGe89 2024/11/23-17:11:02.236634 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst): first pass time = 0.91's, 147.298'MB/sec 2024/11/23-17:11:03.555970 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst): old prealloc_size = 147639500, real_size = 32364016 2024/11/23-17:11:03.556550 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.95 avg-zval = 32.79 2024/11/23-17:11:03.557273 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:11:04.382357 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271497.sst): second pass time = 1.32's, 82.127'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.844'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.376'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 283.428'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35107.400'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.127'MB/sec zip pipeline throughput = 82.127'MB/sec entries = 925987 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 32.79 usrkeys = 925987 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.92 avg-zval = 32.79 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 789707 bytes = 2862704 } seq expand size = 5527949 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4436 value = 3.5670 dict = 1.00 all = 3.7045 } Zip/UnZip{ index = 0.0958 value = 0.2803 dict = 1.00 all = 0.2699 } ---------------------------- total value len = 11.013047 GB avg = 0.157 KB (by entry num) total key len = 1.939164 GB avg = 0.028 KB total ukey len = 1.353837 GB avg = 0.016 KB total ukey num = 0.083348389 Billion total entry num = 0.070156118 Billion write speed all = 0.435336714 MB/sec (with seq num) write speed all = 0.416472578 MB/sec (without seq num) 2024/11/23-17:11:04.426726 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271497: 925987 keys, 36231092 bytes, temperature: kWarm 2024/11/23-17:11:04.426795 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353064426759, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271497, "file_size": 36231092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30359008, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927636, "raw_average_key_size": 28, "raw_value_size": 108290194, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353061, "oldest_key_time": 0, "file_creation_time": 1732353061, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271497, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:04.467872 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst) /dev/shm/ToplingTemp/Topling-127513-FwPwR7 2024/11/23-17:11:05.461650 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst): first pass time = 0.99's, 135.063'MB/sec 2024/11/23-17:11:06.946155 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst): old prealloc_size = 147639500, real_size = 36864080 2024/11/23-17:11:06.946695 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926404 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.88 avg-zval = 37.64 2024/11/23-17:11:06.946908 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:11:07.653618 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271498.sst): second pass time = 1.48's, 72.977'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.314'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.396'MB/sec write SST data time = 0.12's, 313.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32452.194'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.977'MB/sec zip pipeline throughput = 72.977'MB/sec entries = 926404 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 37.64 usrkeys = 926404 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.91 avg-zval = 37.64 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 135696 bytes = 424064 } seq expand size = 949872 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5132 value = 3.1052 dict = 1.00 all = 3.5052 } Zip/UnZip{ index = 0.0951 value = 0.3220 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 11.122275 GB avg = 0.156 KB (by entry num) total key len = 1.965103 GB avg = 0.028 KB total ukey len = 1.372365 GB avg = 0.016 KB total ukey num = 0.084274793 Billion total entry num = 0.071082522 Billion write speed all = 0.439831474 MB/sec (with seq num) write speed all = 0.420720340 MB/sec (without seq num) 2024/11/23-17:11:07.699140 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271498: 926404 keys, 38291261 bytes, temperature: kWarm 2024/11/23-17:11:07.699210 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353067699176, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271498, "file_size": 38291261, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34869952, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939312, "raw_average_key_size": 28, "raw_value_size": 108278456, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926404, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353064, "oldest_key_time": 0, "file_creation_time": 1732353064, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271498, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:07.738871 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst) /dev/shm/ToplingTemp/Topling-127513-wkPhA8 2024/11/23-17:11:08.768230 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst): first pass time = 1.03's, 130.394'MB/sec 2024/11/23-17:11:10.142622 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst): old prealloc_size = 147639500, real_size = 36007136 2024/11/23-17:11:10.143156 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925445 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.03 avg-zval = 36.76 2024/11/23-17:11:10.143697 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:11:10.918217 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271499.sst): second pass time = 1.37's, 78.849'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 50.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3446.492'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.356'MB/sec write SST data time = 0.12's, 313.269'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35787.647'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.849'MB/sec zip pipeline throughput = 78.849'MB/sec entries = 925445 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 36.76 usrkeys = 925445 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.20 avg-zval = 36.76 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 550840 bytes = 2065664 } seq expand size = 3855880 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5118 value = 3.1841 dict = 1.00 all = 3.4348 } Zip/UnZip{ index = 0.0951 value = 0.3141 dict = 1.00 all = 0.2911 } ---------------------------- total value len = 11.234437 GB avg = 0.156 KB (by entry num) total key len = 1.991016 GB avg = 0.028 KB total ukey len = 1.390874 GB avg = 0.016 KB total ukey num = 0.085200238 Billion total entry num = 0.072007967 Billion write speed all = 0.444422997 MB/sec (with seq num) write speed all = 0.425065173 MB/sec (without seq num) 2024/11/23-17:11:10.964968 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271499: 925445 keys, 39076272 bytes, temperature: kWarm 2024/11/23-17:11:10.965053 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353070965016, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271499, "file_size": 39076272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4136223609, "table_properties": {"data_size": 34014800, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912460, "raw_average_key_size": 28, "raw_value_size": 108305298, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925445, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353067, "oldest_key_time": 0, "file_creation_time": 1732353067, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271499, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:11.001833 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst) /dev/shm/ToplingTemp/Topling-127513-mx0fs7 2024/11/23-17:11:11.855957 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst): first pass time = 0.85's, 157.147'MB/sec 2024/11/23-17:11:13.202248 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst): old prealloc_size = 147639500, real_size = 35523968 2024/11/23-17:11:13.202785 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926521 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 116.86 avg-zval = 36.18 2024/11/23-17:11:13.203517 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:11:14.029039 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271500.sst): second pass time = 1.35's, 80.472'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.644'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3440.016'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.12's, 316.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35372.708'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.472'MB/sec zip pipeline throughput = 80.472'MB/sec entries = 926521 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.86 avg-zval = 36.18 usrkeys = 926521 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.86 avg-zval = 36.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926521 bytes = 3011216 } seq expand size = 6485647 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4455 value = 3.2303 dict = 1.00 all = 3.4074 } Zip/UnZip{ index = 0.0957 value = 0.3096 dict = 1.00 all = 0.2935 } ---------------------------- total value len = 11.349197 GB avg = 0.156 KB (by entry num) total key len = 2.016958 GB avg = 0.028 KB total ukey len = 1.409404 GB avg = 0.016 KB total ukey num = 0.086126759 Billion total entry num = 0.072934488 Billion write speed all = 0.449104194 MB/sec (with seq num) write speed all = 0.429499344 MB/sec (without seq num) 2024/11/23-17:11:14.078194 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271500: 926521 keys, 39389752 bytes, temperature: kWarm 2024/11/23-17:11:14.078266 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353074078228, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271500, "file_size": 39389752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4136223669, "table_properties": {"data_size": 33518128, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942588, "raw_average_key_size": 28, "raw_value_size": 108275202, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926521, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353070, "oldest_key_time": 0, "file_creation_time": 1732353070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271500, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:14.114622 139957299091008 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst) /dev/shm/ToplingTemp/Topling-127513-5nnkU8 2024/11/23-17:11:14.189070 139957299091008 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst): first pass time = 0.07's, 235.292'MB/sec 2024/11/23-17:11:14.348935 139957299091008 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst): old prealloc_size = 147639500, real_size = 5147952 2024/11/23-17:11:14.349085 139957299091008 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122054 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0141 GB zip-val = 0.0049 GB avg-val = 115.45 avg-zval = 40.01 2024/11/23-17:11:14.349226 139957299091008 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/23-17:11:14.461301 139957299091008 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00/271501.sst): second pass time = 0.16's, 88.512'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 50.278'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1735.474'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.324'MB/sec write SST data time = 0.02's, 313.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.148'MB, throughput = 5303.011'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 88.512'MB/sec zip pipeline throughput = 88.512'MB/sec entries = 122054 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.45 avg-zval = 40.01 usrkeys = 122054 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.45 avg-zval = 40.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122054 bytes = 381440 } seq expand size = 854378 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0024 GB value = 0.0141 GB dict = 0.15 MB all = 0.0175 GB } __ZipSize{ index = 0.0002 GB value = 0.0049 GB dict = 0.15 MB all = 0.0056 GB } UnZip/Zip{ index = 10.4498 value = 2.8853 dict = 1.00 all = 3.0996 } Zip/UnZip{ index = 0.0957 value = 0.3466 dict = 1.00 all = 0.3226 } ---------------------------- total value len = 11.364143 GB avg = 0.156 KB (by entry num) total key len = 2.020376 GB avg = 0.028 KB total ukey len = 1.411846 GB avg = 0.016 KB total ukey num = 0.086248813 Billion total entry num = 0.073056542 Billion write speed all = 0.449714655 MB/sec (with seq num) write speed all = 0.430077282 MB/sec (without seq num) 2024/11/23-17:11:14.474633 139957299091008 [compaction_job.cc:1948] [default] [JOB 31565] Generated table #271501: 122054 keys, 5648705 bytes, temperature: kWarm 2024/11/23-17:11:14.474695 139957299091008 EVENT_LOG_v1 {"time_micros": 1732353074474665, "cf_name": "default", "job": 31565, "event": "table_file_creation", "file_number": 271501, "file_size": 5648705, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739047, "largest_seqno": 4136223085, "table_properties": {"data_size": 4883680, "index_size": 233600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3417512, "raw_average_key_size": 28, "raw_value_size": 14090960, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 122054, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353074, "oldest_key_time": 0, "file_creation_time": 1732353074, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 271501, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:11:14.485925 139957299091008 [compaction_job.cc:703] job-31565: subcompact[0], size: 0.844526 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31565/att-00 [271479,271480,271481,271482,271483,271484,271485,271486,271487,271488,271489,271490,271491,271492,271493,271494,271495,271496,271497,271498,271499,271500,271501] 2024/11/23-17:11:14.624526 139957299091008 (Original Log Time 2024/11/23-17:09:56.239267) EVENT_LOG_v1 {"time_micros": 1732352996239229, "job": 31565, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [271455], "files_L3": [271376, 271377, 271406, 271407, 271408, 271409, 271410, 271411, 271412, 271413, 271414, 271415, 271416, 271417, 271418, 271419, 271420, 271421, 271422, 271423, 271424, 271474], "score": 0.998536, "input_data_size": 867390328, "oldest_snapshot_seqno": 4136360010} 2024/11/23-17:11:14.624530 139957299091008 (Original Log Time 2024/11/23-17:11:14.622919) [compaction_job.cc:2022] [default] [JOB 31565] Compacted 1@2 + 22@3 files to L3 => 844525829 bytes 2024/11/23-17:11:14.624531 139957299091008 (Original Log Time 2024/11/23-17:11:14.624453) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.8 wr, level 3, files in(1, 22) out(23 +0 blob) MB in(27.1, 800.1 +0.0 blob) out(805.4 +0.0 blob), read-write-amplify(60.2) write-amplify(29.7) OK, records in: 20811147, records dropped: 112769 output_compression: Snappy 2024/11/23-17:11:14.624535 139957299091008 (Original Log Time 2024/11/23-17:11:14.624500) EVENT_LOG_v1 {"time_micros": 1732353074624477, "cf": "default", "job": 31565, "event": "compaction_finished", "compaction_time_micros": 78246664, "compaction_time_cpu_micros": 32641971, "output_level": 3, "num_output_files": 23, "total_output_size": 844525829, "num_input_records": 20811147, "num_output_records": 20698378, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 660264925, "file_range_sync_nanos": 0, "file_fsync_nanos": 16601, "file_prepare_write_nanos": 63058, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:11:14.757227 139957299091008 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732352996,"executesMs":78535,"compactionJobId":31565,"attempt":0,"compactionInputRawBytes":2986313127,"compactionInputZipBytes":867390328,"compactionOutputRawBytes":0,"compactionOutputZipBytes":844525829,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8ab9e7d079f241cbfcd79f7bf717449c-094762c2ac820e6c-00"}}, response = 2024/11/23-17:11:14.757322 139957299091008 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31565/att-00: olev 3, work 78.536 s, result 6.042 ms, install 1.632 ms, input{raw 2.781 GiB zip 827.208 MiB}