2024/11/24-05:00:00.560216 139957517301312 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00: fork to child time = 0.007554 sec 2024/11/24-05:00:00.561081 139957517301312 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00 2024/11/24-05:00:00.567042 139957517301312 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00 2024/11/24-05:00:00.567944 139957517301312 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:00:00.568221 139957517301312 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:00:00.623669 139957517301312 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41899/att-00/MANIFEST-206347 2024/11/24-05:00:00.652702 139957517301312 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41899/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292821, last_sequence is 4450347388, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292623 2024/11/24-05:00:00.652715 139957517301312 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:00:00.652855 139957517301312 [version_set.cc:5721] Creating manifest 292821 2024/11/24-05:00:00.693125 139957517301312 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:00:00.714166 139957517301312 [compaction_job.cc:2437] [default] [JOB 41899] Compacting 1@2 + 24@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/24-05:00:00.714184 139957517301312 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31407 Base level 2, inputs: [292758(27MB)], [292818(35MB) 292819(6512KB) 292721(33MB) 292722(34MB) 292723(35MB) 292724(33MB) 292725(35MB) 292726(35MB) 292727(33MB) 292728(35MB) 292729(35MB) 292730(13MB) 292699(34MB) 292700(33MB) 292701(35MB) 292702(30MB) 292703(31MB) 292704(33MB) 292705(31MB) 292706(35MB) 292707(33MB) 292708(34MB) 292709(33MB) 292790(34MB)] 2024/11/24-05:00:00.722569 139957517301312 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:00:00.722601 139957517301312 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:00:00.722621 139957517301312 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:00:00.722627 139957517301312 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:00:00.722639 139957517301312 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:00:01.592913 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst) /dev/shm/ToplingTemp/Topling-275316-0ucNsi 2024/11/24-05:00:02.153064 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst): first pass time = 0.56's, 239.637'MB/sec 2024/11/24-05:00:03.514823 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst): old prealloc_size = 147639500, real_size = 33147344 2024/11/24-05:00:03.515398 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927035 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.78 avg-zval = 33.60 2024/11/24-05:00:03.516205 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:04.365038 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292822.sst): second pass time = 1.36's, 79.578'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.318'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 293.533'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26617.310'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.578'MB/sec zip pipeline throughput = 79.578'MB/sec entries = 927035 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 33.60 usrkeys = 927035 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 33.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927035 bytes = 3244640 } seq expand size = 6489245 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5119 value = 3.4753 dict = 1.00 all = 3.6035 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 2251.440323 GB avg = 0.200 KB (by entry num) total key len = 306.908919 GB avg = 0.027 KB total ukey len = 237.373268 GB avg = 0.015 KB total ukey num = 15.739588198 Billion total entry num = 11.280902146 Billion write speed all = 35.389032276 MB/sec (with seq num) write speed all = 34.140664138 MB/sec (without seq num) 2024/11/24-05:00:04.411604 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292822: 927035 keys, 37246402 bytes, temperature: kWarm 2024/11/24-05:00:04.411704 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395604411643, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292822, "file_size": 37246402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4450019610, "table_properties": {"data_size": 31151616, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956980, "raw_average_key_size": 28, "raw_value_size": 108260838, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927035, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395601, "oldest_key_time": 0, "file_creation_time": 1732395601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292822, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:04.459700 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst) /dev/shm/ToplingTemp/Topling-275316-dO2rci 2024/11/24-05:00:05.571949 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst): first pass time = 1.11's, 120.677'MB/sec 2024/11/24-05:00:06.888008 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst): old prealloc_size = 147639500, real_size = 31389008 2024/11/24-05:00:06.888586 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921151 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.71 avg-zval = 31.91 2024/11/24-05:00:06.889391 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:07.704028 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292823.sst): second pass time = 1.31's, 82.464'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.737'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 279.002'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31367.470'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.464'MB/sec zip pipeline throughput = 82.464'MB/sec entries = 921151 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.71 avg-zval = 31.91 usrkeys = 921151 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.71 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921151 bytes = 3224048 } seq expand size = 6448057 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4441 value = 3.6886 dict = 1.00 all = 3.7839 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.555197 GB avg = 0.200 KB (by entry num) total key len = 306.934712 GB avg = 0.027 KB total ukey len = 237.391691 GB avg = 0.015 KB total ukey num = 15.740509349 Billion total entry num = 11.281823297 Billion write speed all = 35.389343532 MB/sec (with seq num) write speed all = 34.140931118 MB/sec (without seq num) 2024/11/24-05:00:07.747223 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292823: 921151 keys, 35470356 bytes, temperature: kWarm 2024/11/24-05:00:07.747303 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395607747266, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292823, "file_size": 35470356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4309435005, "largest_seqno": 4450019261, "table_properties": {"data_size": 29394560, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25792228, "raw_average_key_size": 28, "raw_value_size": 108425618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395604, "oldest_key_time": 0, "file_creation_time": 1732395604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:07.838599 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst) /dev/shm/ToplingTemp/Topling-275316-xyUAKg 2024/11/24-05:00:08.764905 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst): first pass time = 0.93's, 144.903'MB/sec 2024/11/24-05:00:10.099667 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst): old prealloc_size = 147639500, real_size = 33033104 2024/11/24-05:00:10.100327 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925318 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.05 avg-zval = 33.55 2024/11/24-05:00:10.101132 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:10.947701 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292824.sst): second pass time = 1.33's, 81.197'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2769.267'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 294.404'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34337.497'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.197'MB/sec zip pipeline throughput = 81.197'MB/sec entries = 925318 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.55 usrkeys = 925318 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925318 bytes = 3238624 } seq expand size = 6477226 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5127 value = 3.4892 dict = 1.00 all = 3.6151 } Zip/UnZip{ index = 0.0951 value = 0.2866 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 2251.669983 GB avg = 0.200 KB (by entry num) total key len = 306.960621 GB avg = 0.027 KB total ukey len = 237.410197 GB avg = 0.015 KB total ukey num = 15.741434667 Billion total entry num = 11.282748615 Billion write speed all = 35.389701810 MB/sec (with seq num) write speed all = 34.141243018 MB/sec (without seq num) 2024/11/24-05:00:10.994544 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292824: 925318 keys, 37126516 bytes, temperature: kWarm 2024/11/24-05:00:10.994618 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395610994581, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292824, "file_size": 37126516, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4450017152, "table_properties": {"data_size": 31041216, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25908904, "raw_average_key_size": 28, "raw_value_size": 108308856, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925318, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395607, "oldest_key_time": 0, "file_creation_time": 1732395607, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:11.037219 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst) /dev/shm/ToplingTemp/Topling-275316-wIt7mk 2024/11/24-05:00:12.072206 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst): first pass time = 1.03's, 129.688'MB/sec 2024/11/24-05:00:13.413106 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst): old prealloc_size = 147639500, real_size = 32763120 2024/11/24-05:00:13.413695 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927258 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.75 avg-zval = 33.17 2024/11/24-05:00:13.414500 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:14.261375 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292825.sst): second pass time = 1.34's, 80.783'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.458'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.275'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 293.486'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33879.535'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.783'MB/sec zip pipeline throughput = 80.783'MB/sec entries = 927258 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 33.17 usrkeys = 927258 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927258 bytes = 3245424 } seq expand size = 6490806 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4440 value = 3.5199 dict = 1.00 all = 3.6410 } Zip/UnZip{ index = 0.0957 value = 0.2841 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 2251.784728 GB avg = 0.200 KB (by entry num) total key len = 306.986584 GB avg = 0.027 KB total ukey len = 237.428743 GB avg = 0.015 KB total ukey num = 15.742361925 Billion total entry num = 11.283675873 Billion write speed all = 35.390025987 MB/sec (with seq num) write speed all = 34.141521815 MB/sec (without seq num) 2024/11/24-05:00:14.306598 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292825: 927258 keys, 36863064 bytes, temperature: kWarm 2024/11/24-05:00:14.306675 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395614306635, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292825, "file_size": 36863064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4450019460, "table_properties": {"data_size": 30755424, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963224, "raw_average_key_size": 28, "raw_value_size": 108254648, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395611, "oldest_key_time": 0, "file_creation_time": 1732395611, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:14.343808 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst) /dev/shm/ToplingTemp/Topling-275316-HGVyDj 2024/11/24-05:00:15.200054 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst): first pass time = 0.86's, 156.758'MB/sec 2024/11/24-05:00:16.501738 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst): old prealloc_size = 147639500, real_size = 31478384 2024/11/24-05:00:16.502356 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-05:00:16.503125 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:17.320648 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292826.sst): second pass time = 1.30's, 83.390'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.419'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2965.268'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 282.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32379.659'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.390'MB/sec zip pipeline throughput = 83.390'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4455 value = 3.6787 dict = 1.00 all = 3.7749 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.899641 GB avg = 0.200 KB (by entry num) total key len = 307.012324 GB avg = 0.027 KB total ukey len = 237.447129 GB avg = 0.015 KB total ukey num = 15.743281225 Billion total entry num = 11.284595173 Billion write speed all = 35.390473884 MB/sec (with seq num) write speed all = 34.141920823 MB/sec (without seq num) 2024/11/24-05:00:17.364330 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292826: 919300 keys, 35555122 bytes, temperature: kWarm 2024/11/24-05:00:17.364403 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395617364367, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292826, "file_size": 35555122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296022951, "largest_seqno": 4447699522, "table_properties": {"data_size": 29488160, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395614, "oldest_key_time": 0, "file_creation_time": 1732395614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:17.402757 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst) /dev/shm/ToplingTemp/Topling-275316-Md2gQi 2024/11/24-05:00:18.430815 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst): first pass time = 1.03's, 130.560'MB/sec 2024/11/24-05:00:19.775492 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst): old prealloc_size = 147639500, real_size = 33130720 2024/11/24-05:00:19.776082 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927251 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.75 avg-zval = 33.58 2024/11/24-05:00:19.776861 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:20.631160 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292827.sst): second pass time = 1.34's, 80.550'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3094.895'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 296.177'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32436.276'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.550'MB/sec zip pipeline throughput = 80.550'MB/sec entries = 927251 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.58 usrkeys = 927251 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927251 bytes = 3245392 } seq expand size = 6490757 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5113 value = 3.4770 dict = 1.00 all = 3.6051 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 2252.014387 GB avg = 0.200 KB (by entry num) total key len = 307.038287 GB avg = 0.027 KB total ukey len = 237.465674 GB avg = 0.015 KB total ukey num = 15.744208476 Billion total entry num = 11.285522424 Billion write speed all = 35.390799542 MB/sec (with seq num) write speed all = 34.142201056 MB/sec (without seq num) 2024/11/24-05:00:20.678698 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292827: 927251 keys, 37230224 bytes, temperature: kWarm 2024/11/24-05:00:20.678770 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395620678734, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292827, "file_size": 37230224, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4450019287, "table_properties": {"data_size": 31134416, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963028, "raw_average_key_size": 28, "raw_value_size": 108254770, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927251, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395617, "oldest_key_time": 0, "file_creation_time": 1732395617, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:20.718950 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst) /dev/shm/ToplingTemp/Topling-275316-zdGR9j 2024/11/24-05:00:21.557316 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst): first pass time = 0.84's, 160.106'MB/sec 2024/11/24-05:00:22.887844 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst): old prealloc_size = 147639500, real_size = 31884800 2024/11/24-05:00:22.888468 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927571 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.70 avg-zval = 32.21 2024/11/24-05:00:22.889251 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:23.717276 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292828.sst): second pass time = 1.33's, 81.400'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.387'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2970.127'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 286.838'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32649.601'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.400'MB/sec zip pipeline throughput = 81.400'MB/sec entries = 927571 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.70 avg-zval = 32.21 usrkeys = 927571 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.70 avg-zval = 32.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927571 bytes = 3246512 } seq expand size = 6492997 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4437 value = 3.6231 dict = 1.00 all = 3.7298 } Zip/UnZip{ index = 0.0958 value = 0.2760 dict = 1.00 all = 0.2681 } ---------------------------- total value len = 2252.129125 GB avg = 0.200 KB (by entry num) total key len = 307.064259 GB avg = 0.027 KB total ukey len = 237.484225 GB avg = 0.015 KB total ukey num = 15.745136047 Billion total entry num = 11.286449995 Billion write speed all = 35.391235033 MB/sec (with seq num) write speed all = 34.142587215 MB/sec (without seq num) 2024/11/24-05:00:23.761650 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292828: 927571 keys, 35985414 bytes, temperature: kWarm 2024/11/24-05:00:23.761726 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395623761688, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292828, "file_size": 35985414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4450019300, "table_properties": {"data_size": 29876400, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971988, "raw_average_key_size": 28, "raw_value_size": 108245854, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927571, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395620, "oldest_key_time": 0, "file_creation_time": 1732395620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:23.798016 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst) /dev/shm/ToplingTemp/Topling-275316-VoyIrk 2024/11/24-05:00:24.739468 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst): first pass time = 0.94's, 142.570'MB/sec 2024/11/24-05:00:26.049996 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst): old prealloc_size = 147639500, real_size = 31464800 2024/11/24-05:00:26.050584 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.06 2024/11/24-05:00:26.051358 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:26.863193 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292829.sst): second pass time = 1.31's, 82.854'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.706'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.453'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 285.835'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32854.201'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.854'MB/sec zip pipeline throughput = 82.854'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.06 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4466 value = 3.6803 dict = 1.00 all = 3.7764 } Zip/UnZip{ index = 0.0957 value = 0.2717 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2252.244038 GB avg = 0.200 KB (by entry num) total key len = 307.090000 GB avg = 0.027 KB total ukey len = 237.502611 GB avg = 0.015 KB total ukey num = 15.746055347 Billion total entry num = 11.287369295 Billion write speed all = 35.391640411 MB/sec (with seq num) write speed all = 34.142945214 MB/sec (without seq num) 2024/11/24-05:00:26.908607 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292829: 919300 keys, 35541490 bytes, temperature: kWarm 2024/11/24-05:00:26.908680 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395626908643, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292829, "file_size": 35541490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280305501, "largest_seqno": 4432343913, "table_properties": {"data_size": 29474768, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395623, "oldest_key_time": 0, "file_creation_time": 1732395623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:26.945426 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst) /dev/shm/ToplingTemp/Topling-275316-rIM1Qj 2024/11/24-05:00:27.972222 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst): first pass time = 1.03's, 130.720'MB/sec 2024/11/24-05:00:29.368553 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst): old prealloc_size = 147639500, real_size = 33464464 2024/11/24-05:00:29.369175 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927232 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.75 avg-zval = 33.94 2024/11/24-05:00:29.370924 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:30.223967 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292830.sst): second pass time = 1.40's, 77.573'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2944.888'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 299.927'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33886.700'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.573'MB/sec zip pipeline throughput = 77.573'MB/sec entries = 927232 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.94 usrkeys = 927232 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927232 bytes = 3245328 } seq expand size = 6490624 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5126 value = 3.4401 dict = 1.00 all = 3.5730 } Zip/UnZip{ index = 0.0951 value = 0.2907 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 2252.358784 GB avg = 0.200 KB (by entry num) total key len = 307.115962 GB avg = 0.027 KB total ukey len = 237.521156 GB avg = 0.015 KB total ukey num = 15.746982579 Billion total entry num = 11.288296527 Billion write speed all = 35.391941375 MB/sec (with seq num) write speed all = 34.143201634 MB/sec (without seq num) 2024/11/24-05:00:30.269680 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292830: 927232 keys, 37564100 bytes, temperature: kWarm 2024/11/24-05:00:30.269748 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395630269715, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292830, "file_size": 37564100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4450019311, "table_properties": {"data_size": 31468416, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962496, "raw_average_key_size": 28, "raw_value_size": 108255256, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927232, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395626, "oldest_key_time": 0, "file_creation_time": 1732395626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:30.312400 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst) /dev/shm/ToplingTemp/Topling-275316-ObRaSg 2024/11/24-05:00:31.249513 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst): first pass time = 0.94's, 143.230'MB/sec 2024/11/24-05:00:32.567962 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst): old prealloc_size = 147639500, real_size = 31404192 2024/11/24-05:00:32.568544 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927319 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 116.74 avg-zval = 31.70 2024/11/24-05:00:32.569354 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:33.394593 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292831.sst): second pass time = 1.32's, 82.174'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.015'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 280.307'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35032.620'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.174'MB/sec zip pipeline throughput = 82.174'MB/sec entries = 927319 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 31.70 usrkeys = 927319 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.74 avg-zval = 31.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927319 bytes = 3245632 } seq expand size = 6491233 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4462 value = 3.6825 dict = 1.00 all = 3.7803 } Zip/UnZip{ index = 0.0957 value = 0.2716 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 2252.473528 GB avg = 0.200 KB (by entry num) total key len = 307.141927 GB avg = 0.027 KB total ukey len = 237.539702 GB avg = 0.015 KB total ukey num = 15.747909898 Billion total entry num = 11.289223846 Billion write speed all = 35.392335372 MB/sec (with seq num) write speed all = 34.143547800 MB/sec (without seq num) 2024/11/24-05:00:33.438104 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292831: 927319 keys, 35504230 bytes, temperature: kWarm 2024/11/24-05:00:33.438174 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395633438140, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292831, "file_size": 35504230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4450019324, "table_properties": {"data_size": 29396752, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964932, "raw_average_key_size": 28, "raw_value_size": 108252810, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927319, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395630, "oldest_key_time": 0, "file_creation_time": 1732395630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:33.479094 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst) /dev/shm/ToplingTemp/Topling-275316-2EPqqi 2024/11/24-05:00:34.596182 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst): first pass time = 1.12's, 120.154'MB/sec 2024/11/24-05:00:35.909008 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst): old prealloc_size = 147639500, real_size = 31448064 2024/11/24-05:00:35.909589 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.04 2024/11/24-05:00:35.910350 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:36.727615 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292832.sst): second pass time = 1.31's, 82.684'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.337'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 280.004'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33758.867'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.684'MB/sec zip pipeline throughput = 82.684'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.04 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6825 dict = 1.00 all = 3.7781 } Zip/UnZip{ index = 0.0957 value = 0.2716 dict = 1.00 all = 0.2647 } ---------------------------- total value len = 2252.588440 GB avg = 0.200 KB (by entry num) total key len = 307.167667 GB avg = 0.027 KB total ukey len = 237.558088 GB avg = 0.015 KB total ukey num = 15.748829198 Billion total entry num = 11.290143146 Billion write speed all = 35.392649090 MB/sec (with seq num) write speed all = 34.143817380 MB/sec (without seq num) 2024/11/24-05:00:36.770697 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292832: 919300 keys, 35524786 bytes, temperature: kWarm 2024/11/24-05:00:36.770769 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395636770735, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292832, "file_size": 35524786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4274751581, "largest_seqno": 4427241104, "table_properties": {"data_size": 29457648, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395633, "oldest_key_time": 0, "file_creation_time": 1732395633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:36.809752 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst) /dev/shm/ToplingTemp/Topling-275316-d7qhhl 2024/11/24-05:00:37.812702 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst): first pass time = 1.00's, 133.828'MB/sec 2024/11/24-05:00:39.172008 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst): old prealloc_size = 147639500, real_size = 33337968 2024/11/24-05:00:39.172571 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927014 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.79 avg-zval = 33.81 2024/11/24-05:00:39.173371 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:40.030290 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292833.sst): second pass time = 1.36's, 79.690'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.619'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.204'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 297.324'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36618.707'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.690'MB/sec zip pipeline throughput = 79.690'MB/sec entries = 927014 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.81 usrkeys = 927014 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927014 bytes = 3244560 } seq expand size = 6489098 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5159 value = 3.4541 dict = 1.00 all = 3.5852 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 2252.703191 GB avg = 0.200 KB (by entry num) total key len = 307.193624 GB avg = 0.027 KB total ukey len = 237.576628 GB avg = 0.015 KB total ukey num = 15.749756212 Billion total entry num = 11.291070160 Billion write speed all = 35.392978377 MB/sec (with seq num) write speed all = 34.144101158 MB/sec (without seq num) 2024/11/24-05:00:40.075602 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292833: 927014 keys, 37436766 bytes, temperature: kWarm 2024/11/24-05:00:40.075669 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395640075635, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292833, "file_size": 37436766, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4450019344, "table_properties": {"data_size": 31342944, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956392, "raw_average_key_size": 28, "raw_value_size": 108261356, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927014, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395636, "oldest_key_time": 0, "file_creation_time": 1732395636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:40.113041 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst) /dev/shm/ToplingTemp/Topling-275316-JB7k5h 2024/11/24-05:00:41.142303 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst): first pass time = 1.03's, 130.407'MB/sec 2024/11/24-05:00:42.459518 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst): old prealloc_size = 147639500, real_size = 31031584 2024/11/24-05:00:42.460048 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927414 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 116.72 avg-zval = 31.30 2024/11/24-05:00:42.460804 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:43.279714 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292834.sst): second pass time = 1.32's, 82.231'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.840'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3525.306'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.920'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33033.791'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.231'MB/sec zip pipeline throughput = 82.231'MB/sec entries = 927414 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.72 avg-zval = 31.30 usrkeys = 927414 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.72 avg-zval = 31.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927414 bytes = 3245968 } seq expand size = 6491898 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4439 value = 3.7297 dict = 1.00 all = 3.8200 } Zip/UnZip{ index = 0.0958 value = 0.2681 dict = 1.00 all = 0.2618 } ---------------------------- total value len = 2252.817933 GB avg = 0.200 KB (by entry num) total key len = 307.219591 GB avg = 0.027 KB total ukey len = 237.595177 GB avg = 0.015 KB total ukey num = 15.750683626 Billion total entry num = 11.291997574 Billion write speed all = 35.393333725 MB/sec (with seq num) write speed all = 34.144410036 MB/sec (without seq num) 2024/11/24-05:00:43.333586 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292834: 927414 keys, 35135614 bytes, temperature: kWarm 2024/11/24-05:00:43.333659 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395643333623, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292834, "file_size": 35135614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4450019361, "table_properties": {"data_size": 29023568, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967592, "raw_average_key_size": 28, "raw_value_size": 108250144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927414, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395640, "oldest_key_time": 0, "file_creation_time": 1732395640, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:43.378379 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst) /dev/shm/ToplingTemp/Topling-275316-mVO4wi 2024/11/24-05:00:44.223908 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst): first pass time = 0.85's, 158.745'MB/sec 2024/11/24-05:00:45.526702 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst): old prealloc_size = 147639500, real_size = 30860224 2024/11/24-05:00:45.527299 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926621 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.85 avg-zval = 31.13 2024/11/24-05:00:45.528096 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:46.346907 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292835.sst): second pass time = 1.30's, 83.182'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.620'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3093.804'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 274.644'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32912.352'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.182'MB/sec zip pipeline throughput = 83.182'MB/sec entries = 926621 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.85 avg-zval = 31.13 usrkeys = 926621 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.85 avg-zval = 31.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926621 bytes = 3243184 } seq expand size = 6486347 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3781 value = 3.7539 dict = 1.00 all = 3.8396 } Zip/UnZip{ index = 0.0964 value = 0.2664 dict = 1.00 all = 0.2604 } ---------------------------- total value len = 2252.932692 GB avg = 0.199 KB (by entry num) total key len = 307.245537 GB avg = 0.027 KB total ukey len = 237.613709 GB avg = 0.015 KB total ukey num = 15.751610247 Billion total entry num = 11.292924195 Billion write speed all = 35.393778135 MB/sec (with seq num) write speed all = 34.144804922 MB/sec (without seq num) 2024/11/24-05:00:46.392093 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292835: 926621 keys, 34956374 bytes, temperature: kWarm 2024/11/24-05:00:46.392163 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395646392128, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292835, "file_size": 34956374, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4450016667, "table_properties": {"data_size": 28842672, "index_size": 1785728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25945388, "raw_average_key_size": 28, "raw_value_size": 108272446, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926621, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395643, "oldest_key_time": 0, "file_creation_time": 1732395643, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292835, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:46.434099 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst) /dev/shm/ToplingTemp/Topling-275316-aTYjhh 2024/11/24-05:00:47.377581 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst): first pass time = 0.94's, 142.265'MB/sec 2024/11/24-05:00:48.631365 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst): old prealloc_size = 147639500, real_size = 27769632 2024/11/24-05:00:48.631898 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.21 2024/11/24-05:00:48.632714 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:49.472734 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292836.sst): second pass time = 1.25's, 85.998'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3557.436'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 245.986'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 31633.631'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 85.998'MB/sec zip pipeline throughput = 85.998'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.21 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4450 value = 4.1889 dict = 1.00 all = 4.2206 } Zip/UnZip{ index = 0.0957 value = 0.2387 dict = 1.00 all = 0.2369 } ---------------------------- total value len = 2253.047060 GB avg = 0.199 KB (by entry num) total key len = 307.272002 GB avg = 0.027 KB total ukey len = 237.632613 GB avg = 0.015 KB total ukey num = 15.752555443 Billion total entry num = 11.293869391 Billion write speed all = 35.394195622 MB/sec (with seq num) write speed all = 34.145171847 MB/sec (without seq num) 2024/11/24-05:00:49.514333 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292836: 945196 keys, 31800816 bytes, temperature: kWarm 2024/11/24-05:00:49.514404 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395649514368, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292836, "file_size": 31800816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4293343940, "largest_seqno": 4394652145, "table_properties": {"data_size": 25723280, "index_size": 1809856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395646, "oldest_key_time": 0, "file_creation_time": 1732395646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292836, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:49.549262 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst) /dev/shm/ToplingTemp/Topling-275316-F9JTXi 2024/11/24-05:00:50.445815 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst): first pass time = 0.90's, 149.712'MB/sec 2024/11/24-05:00:51.805383 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst): old prealloc_size = 147639500, real_size = 32595536 2024/11/24-05:00:51.805956 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933119 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.84 avg-zval = 32.78 2024/11/24-05:00:51.806769 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:52.667340 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292837.sst): second pass time = 1.36's, 79.552'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.593'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.549'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32767.930'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.552'MB/sec zip pipeline throughput = 79.552'MB/sec entries = 933119 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.84 avg-zval = 32.78 usrkeys = 933119 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.84 avg-zval = 32.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933119 bytes = 3265936 } seq expand size = 6531833 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5123 value = 3.5339 dict = 1.00 all = 3.6560 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 2253.161683 GB avg = 0.199 KB (by entry num) total key len = 307.298130 GB avg = 0.027 KB total ukey len = 237.651275 GB avg = 0.015 KB total ukey num = 15.753488562 Billion total entry num = 11.294802510 Billion write speed all = 35.394578238 MB/sec (with seq num) write speed all = 34.145506429 MB/sec (without seq num) 2024/11/24-05:00:52.713858 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292837: 933119 keys, 36711834 bytes, temperature: kWarm 2024/11/24-05:00:52.713930 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395652713894, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292837, "file_size": 36711834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4450016237, "table_properties": {"data_size": 30586752, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127332, "raw_average_key_size": 28, "raw_value_size": 108090442, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933119, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395649, "oldest_key_time": 0, "file_creation_time": 1732395649, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292837, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:52.751801 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst) /dev/shm/ToplingTemp/Topling-275316-WOSivh 2024/11/24-05:00:53.672164 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst): first pass time = 0.92's, 145.837'MB/sec 2024/11/24-05:00:54.999569 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst): old prealloc_size = 147639500, real_size = 31069424 2024/11/24-05:00:55.000154 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924316 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.21 avg-zval = 31.44 2024/11/24-05:00:55.000920 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:55.815136 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292838.sst): second pass time = 1.33's, 81.694'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.970'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.146'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 275.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36892.178'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.694'MB/sec zip pipeline throughput = 81.694'MB/sec entries = 924316 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.21 avg-zval = 31.44 usrkeys = 924316 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.21 avg-zval = 31.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924316 bytes = 3119584 } seq expand size = 6470212 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3761 value = 3.7285 dict = 1.00 all = 3.8300 } Zip/UnZip{ index = 0.0964 value = 0.2682 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 2253.276490 GB avg = 0.199 KB (by entry num) total key len = 307.324011 GB avg = 0.027 KB total ukey len = 237.669762 GB avg = 0.015 KB total ukey num = 15.754412878 Billion total entry num = 11.295726826 Billion write speed all = 35.394982875 MB/sec (with seq num) write speed all = 34.145863202 MB/sec (without seq num) 2024/11/24-05:00:55.859732 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292838: 924316 keys, 35043836 bytes, temperature: kWarm 2024/11/24-05:00:55.859802 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395655859767, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292838, "file_size": 35043836, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4330244476, "largest_seqno": 4450017576, "table_properties": {"data_size": 29056544, "index_size": 1781632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880848, "raw_average_key_size": 28, "raw_value_size": 108336924, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924316, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395652, "oldest_key_time": 0, "file_creation_time": 1732395652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292838, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:55.897502 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst) /dev/shm/ToplingTemp/Topling-275316-ByPXBk 2024/11/24-05:00:56.767245 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst): first pass time = 0.87's, 154.326'MB/sec 2024/11/24-05:00:58.050708 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst): old prealloc_size = 147639500, real_size = 28240464 2024/11/24-05:00:58.051300 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944306 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0262 GB avg-val = 114.13 avg-zval = 27.75 2024/11/24-05:00:58.052109 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:00:58.890015 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292839.sst): second pass time = 1.28's, 84.028'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.592'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.924'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 250.189'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 33894.541'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.028'MB/sec zip pipeline throughput = 84.028'MB/sec entries = 944306 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.13 avg-zval = 27.75 usrkeys = 944306 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.13 avg-zval = 27.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944306 bytes = 3305088 } seq expand size = 6610142 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0262 GB dict = 1.07 MB all = 0.0324 GB } UnZip/Zip{ index = 10.5121 value = 4.1124 dict = 1.00 all = 4.1439 } Zip/UnZip{ index = 0.0951 value = 0.2432 dict = 1.00 all = 0.2413 } ---------------------------- total value len = 2253.390877 GB avg = 0.199 KB (by entry num) total key len = 307.350451 GB avg = 0.027 KB total ukey len = 237.688648 GB avg = 0.015 KB total ukey num = 15.755357184 Billion total entry num = 11.296671132 Billion write speed all = 35.395425096 MB/sec (with seq num) write speed all = 34.146254092 MB/sec (without seq num) 2024/11/24-05:00:58.932588 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292839: 944306 keys, 32388882 bytes, temperature: kWarm 2024/11/24-05:00:58.932668 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395658932631, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292839, "file_size": 32388882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4450018502, "table_properties": {"data_size": 26207616, "index_size": 1796608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26440568, "raw_average_key_size": 28, "raw_value_size": 107777232, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944306, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395655, "oldest_key_time": 0, "file_creation_time": 1732395655, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292839, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:00:58.969034 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst) /dev/shm/ToplingTemp/Topling-275316-M2IGni 2024/11/24-05:00:59.827217 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst): first pass time = 0.86's, 156.404'MB/sec 2024/11/24-05:01:01.120504 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst): old prealloc_size = 147639500, real_size = 31390352 2024/11/24-05:01:01.121062 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-05:01:01.121817 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:01:01.950776 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292840.sst): second pass time = 1.29's, 83.925'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3324.899'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 264.998'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27756.005'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.925'MB/sec zip pipeline throughput = 83.925'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.94 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.94 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3116 value = 3.6926 dict = 1.00 all = 3.7966 } Zip/UnZip{ index = 0.0970 value = 0.2708 dict = 1.00 all = 0.2634 } ---------------------------- total value len = 2253.505790 GB avg = 0.199 KB (by entry num) total key len = 307.376191 GB avg = 0.027 KB total ukey len = 237.707034 GB avg = 0.015 KB total ukey num = 15.756276484 Billion total entry num = 11.297590432 Billion write speed all = 35.395871753 MB/sec (with seq num) write speed all = 34.146651946 MB/sec (without seq num) 2024/11/24-05:01:01.995564 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292840: 919300 keys, 35352242 bytes, temperature: kWarm 2024/11/24-05:01:01.995635 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395661995600, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292840, "file_size": 35352242, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269369607, "largest_seqno": 4385400745, "table_properties": {"data_size": 29377280, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395658, "oldest_key_time": 0, "file_creation_time": 1732395658, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292840, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:01:02.047659 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst) /dev/shm/ToplingTemp/Topling-275316-3PEKzh 2024/11/24-05:01:02.997518 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst): first pass time = 0.95's, 141.309'MB/sec 2024/11/24-05:01:04.315628 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst): old prealloc_size = 147639500, real_size = 30143280 2024/11/24-05:01:04.316210 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931335 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0281 GB avg-val = 116.11 avg-zval = 30.20 2024/11/24-05:01:04.316958 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:01:05.147580 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292841.sst): second pass time = 1.32's, 82.118'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.893'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.826'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 262.668'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27984.220'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.118'MB/sec zip pipeline throughput = 82.118'MB/sec entries = 931335 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.11 avg-zval = 30.20 usrkeys = 931335 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.11 avg-zval = 30.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931335 bytes = 3026848 } seq expand size = 6519345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0340 GB } UnZip/Zip{ index = 10.4413 value = 3.8448 dict = 1.00 all = 3.9453 } Zip/UnZip{ index = 0.0958 value = 0.2601 dict = 1.00 all = 0.2535 } ---------------------------- total value len = 2253.620449 GB avg = 0.199 KB (by entry num) total key len = 307.402269 GB avg = 0.027 KB total ukey len = 237.725660 GB avg = 0.015 KB total ukey num = 15.757207819 Billion total entry num = 11.298521767 Billion write speed all = 35.396252989 MB/sec (with seq num) write speed all = 34.146985399 MB/sec (without seq num) 2024/11/24-05:01:05.191532 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292841: 931335 keys, 34019524 bytes, temperature: kWarm 2024/11/24-05:01:05.191604 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395665191568, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292841, "file_size": 34019524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4383110775, "largest_seqno": 4450018298, "table_properties": {"data_size": 28126304, "index_size": 1783936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077380, "raw_average_key_size": 28, "raw_value_size": 108140446, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395662, "oldest_key_time": 0, "file_creation_time": 1732395662, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292841, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:01:05.236127 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst) /dev/shm/ToplingTemp/Topling-275316-6blt9g 2024/11/24-05:01:06.078881 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst): first pass time = 0.84's, 159.271'MB/sec 2024/11/24-05:01:07.426479 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst): old prealloc_size = 147639500, real_size = 32785744 2024/11/24-05:01:07.427041 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924138 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.24 avg-zval = 33.31 2024/11/24-05:01:07.427822 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:01:08.273933 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292842.sst): second pass time = 1.35's, 80.443'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.516'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.418'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 288.491'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32992.039'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.443'MB/sec zip pipeline throughput = 80.443'MB/sec entries = 924138 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.24 avg-zval = 33.31 usrkeys = 924138 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.24 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924138 bytes = 3234496 } seq expand size = 6468966 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4446 value = 3.5193 dict = 1.00 all = 3.6396 } Zip/UnZip{ index = 0.0957 value = 0.2841 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 2253.735260 GB avg = 0.199 KB (by entry num) total key len = 307.428145 GB avg = 0.027 KB total ukey len = 237.744143 GB avg = 0.015 KB total ukey num = 15.758131957 Billion total entry num = 11.299445905 Billion write speed all = 35.396667958 MB/sec (with seq num) write speed all = 34.147352170 MB/sec (without seq num) 2024/11/24-05:01:08.320664 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292842: 924138 keys, 36876602 bytes, temperature: kWarm 2024/11/24-05:01:08.320735 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395668320698, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292842, "file_size": 36876602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4450018966, "table_properties": {"data_size": 30784896, "index_size": 1769600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875864, "raw_average_key_size": 28, "raw_value_size": 108342000, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395665, "oldest_key_time": 0, "file_creation_time": 1732395665, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292842, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:01:08.361504 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst) /dev/shm/ToplingTemp/Topling-275316-geP86j 2024/11/24-05:01:09.279561 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst): first pass time = 0.92's, 146.204'MB/sec 2024/11/24-05:01:10.595319 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst): old prealloc_size = 147639500, real_size = 31040352 2024/11/24-05:01:10.595859 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923081 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.40 avg-zval = 31.46 2024/11/24-05:01:10.596638 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:01:11.414267 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292843.sst): second pass time = 1.31's, 82.418'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.754'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3426.652'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 271.015'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33758.456'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.418'MB/sec zip pipeline throughput = 82.418'MB/sec entries = 923081 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.40 avg-zval = 31.46 usrkeys = 923081 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.40 avg-zval = 31.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923081 bytes = 3115408 } seq expand size = 6461567 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4463 value = 3.7315 dict = 1.00 all = 3.8335 } Zip/UnZip{ index = 0.0957 value = 0.2680 dict = 1.00 all = 0.2609 } ---------------------------- total value len = 2253.850093 GB avg = 0.199 KB (by entry num) total key len = 307.453991 GB avg = 0.027 KB total ukey len = 237.762605 GB avg = 0.015 KB total ukey num = 15.759055038 Billion total entry num = 11.300368986 Billion write speed all = 35.397075955 MB/sec (with seq num) write speed all = 34.147712330 MB/sec (without seq num) 2024/11/24-05:01:11.458410 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292843: 923081 keys, 35011846 bytes, temperature: kWarm 2024/11/24-05:01:11.458479 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395671458442, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292843, "file_size": 35011846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4353756424, "largest_seqno": 4450018317, "table_properties": {"data_size": 29042128, "index_size": 1767296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846268, "raw_average_key_size": 28, "raw_value_size": 108371582, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923081, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395668, "oldest_key_time": 0, "file_creation_time": 1732395668, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292843, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:01:11.498280 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst) /dev/shm/ToplingTemp/Topling-275316-HsAG3g 2024/11/24-05:01:11.854844 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst): first pass time = 0.36's, 245.056'MB/sec 2024/11/24-05:01:12.743835 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst): old prealloc_size = 147639500, real_size = 22370496 2024/11/24-05:01:12.744261 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 607813 prefix = 4 raw-key = 0.0097 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0703 GB zip-val = 0.0211 GB avg-val = 115.74 avg-zval = 34.65 2024/11/24-05:01:12.744826 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.087 GB 2024/11/24-05:01:13.294298 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00/292844.sst): second pass time = 0.89's, 79.202'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.55's, 45.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2888.911'MB/sec (index lex order gen) rebuild zvType time = 0.47's, 0.326'MB/sec write SST data time = 0.08's, 296.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.705'MB, throughput = 17578.029'MB/sec zip my value time = 0.89's, unzip length = 0.070'GB zip my value throughput = 79.202'MB/sec zip pipeline throughput = 79.202'MB/sec entries = 607813 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.74 avg-zval = 34.65 usrkeys = 607813 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.74 avg-zval = 34.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 607813 bytes = 2127360 } seq expand size = 4254691 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0122 GB value = 0.0703 GB dict = 0.71 MB all = 0.0874 GB } __ZipSize{ index = 0.0012 GB value = 0.0211 GB dict = 0.71 MB all = 0.0251 GB } UnZip/Zip{ index = 10.5074 value = 3.3402 dict = 1.00 all = 3.4870 } Zip/UnZip{ index = 0.0952 value = 0.2994 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 2253.924698 GB avg = 0.199 KB (by entry num) total key len = 307.471010 GB avg = 0.027 KB total ukey len = 237.774761 GB avg = 0.015 KB total ukey num = 15.759662851 Billion total entry num = 11.300976799 Billion write speed all = 35.397422484 MB/sec (with seq num) write speed all = 34.148024122 MB/sec (without seq num) 2024/11/24-05:01:13.328103 139957517301312 [compaction_job.cc:1948] [default] [JOB 41899] Generated table #292844: 607813 keys, 25055496 bytes, temperature: kWarm 2024/11/24-05:01:13.328169 139957517301312 EVENT_LOG_v1 {"time_micros": 1732395673328135, "cf_name": "default", "job": 41899, "event": "table_file_creation", "file_number": 292844, "file_size": 25055496, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4450019034, "table_properties": {"data_size": 21061424, "index_size": 1156928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17018764, "raw_average_key_size": 28, "raw_value_size": 70349482, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 607813, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395671, "oldest_key_time": 0, "file_creation_time": 1732395671, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292844, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:01:13.348011 139957517301312 [compaction_job.cc:703] job-41899: subcompact[0], size: 0.809402 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41899/att-00 [292822,292823,292824,292825,292826,292827,292828,292829,292830,292831,292832,292833,292834,292835,292836,292837,292838,292839,292840,292841,292842,292843,292844] 2024/11/24-05:01:13.480524 139957517301312 (Original Log Time 2024/11/24-05:00:00.714254) EVENT_LOG_v1 {"time_micros": 1732395600714202, "job": 41899, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292758], "files_L3": [292818, 292819, 292721, 292722, 292723, 292724, 292725, 292726, 292727, 292728, 292729, 292730, 292699, 292700, 292701, 292702, 292703, 292704, 292705, 292706, 292707, 292708, 292709, 292790], "score": 0.969154, "input_data_size": 837875420, "oldest_snapshot_seqno": 4450347388} 2024/11/24-05:01:13.480528 139957517301312 (Original Log Time 2024/11/24-05:01:13.478853) [compaction_job.cc:2022] [default] [JOB 41899] Compacted 1@2 + 24@3 files to L3 => 809401536 bytes 2024/11/24-05:01:13.480529 139957517301312 (Original Log Time 2024/11/24-05:01:13.480447) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 24) out(23 +0 blob) MB in(27.1, 772.0 +0.0 blob) out(771.9 +0.0 blob), read-write-amplify(58.0) write-amplify(28.5) OK, records in: 21108152, records dropped: 106464 output_compression: Snappy 2024/11/24-05:01:13.480533 139957517301312 (Original Log Time 2024/11/24-05:01:13.480497) EVENT_LOG_v1 {"time_micros": 1732395673480472, "cf": "default", "job": 41899, "event": "compaction_finished", "compaction_time_micros": 72633766, "compaction_time_cpu_micros": 31408902, "output_level": 3, "num_output_files": 23, "total_output_size": 809401536, "num_input_records": 21108152, "num_output_records": 21001688, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 560701679, "file_range_sync_nanos": 0, "file_fsync_nanos": 17740, "file_prepare_write_nanos": 55249, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/24-05:01:13.621071 139957517301312 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732395600,"executesMs":72910,"compactionJobId":41899,"attempt":0,"compactionInputRawBytes":3055277800,"compactionInputZipBytes":837875420,"compactionOutputRawBytes":0,"compactionOutputZipBytes":809401536,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6ca8f2431329c10cbd6b7e35c557f264-3cc2527c9174e714-00"}}, response = 2024/11/24-05:01:13.621160 139957517301312 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41899/att-00: olev 3, work 72.911 s, result 7.781 ms, install 1.699 ms, input{raw 2.845 GiB zip 799.060 MiB}