2024/11/24-07:22:41.233238 139957198378560 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00: fork to child time = 0.007719 sec 2024/11/24-07:22:41.234064 139957198378560 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00 2024/11/24-07:22:41.239701 139957198378560 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00 2024/11/24-07:22:41.240594 139957198378560 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-07:22:41.240865 139957198378560 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-07:22:41.296838 139957198378560 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43370/att-00/MANIFEST-206347 2024/11/24-07:22:41.322300 139957198378560 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43370/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 296200, last_sequence is 4495256163, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 296005 2024/11/24-07:22:41.322307 139957198378560 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-07:22:41.322451 139957198378560 [version_set.cc:5721] Creating manifest 296200 2024/11/24-07:22:41.361359 139957198378560 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-07:22:41.381648 139957198378560 [compaction_job.cc:2437] [default] [JOB 43370] Compacting 1@2 + 27@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/24-07:22:41.381668 139957198378560 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32507 Base level 2, inputs: [296140(27MB)], [296181(9349KB) 296072(33MB) 296073(35MB) 296074(33MB) 296075(35MB) 296076(33MB) 296077(35MB) 296078(33MB) 296079(34MB) 296080(33MB) 296081(33MB) 296082(34MB) 296083(33MB) 296084(35MB) 296085(33MB) 296087(35MB) 296088(32MB) 296089(30MB) 296090(34MB) 296091(33MB) 296092(31MB) 296093(35MB) 296094(33MB) 296095(32MB) 296096(34MB) 296097(33MB) 296182(34MB)] 2024/11/24-07:22:41.388283 139957198378560 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:22:41.388312 139957198378560 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:22:41.388332 139957198378560 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:22:41.388337 139957198378560 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-07:22:41.388348 139957198378560 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-07:22:41.874061 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst) /dev/shm/ToplingTemp/Topling-296981-8JhMfw 2024/11/24-07:22:42.843930 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst): first pass time = 0.97's, 138.393'MB/sec 2024/11/24-07:22:44.168658 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst): old prealloc_size = 147639500, real_size = 33288704 2024/11/24-07:22:44.169253 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922119 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.55 avg-zval = 33.95 2024/11/24-07:22:44.170075 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:22:45.004666 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296201.sst): second pass time = 1.32's, 81.888'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.707'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3061.778'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 289.596'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30228.052'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.888'MB/sec zip pipeline throughput = 81.888'MB/sec entries = 922119 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 33.95 usrkeys = 922119 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.55 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922119 bytes = 3227424 } seq expand size = 6454833 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5126 value = 3.4628 dict = 1.00 all = 3.5912 } Zip/UnZip{ index = 0.0951 value = 0.2888 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 2251.440427 GB avg = 0.200 KB (by entry num) total key len = 306.908782 GB avg = 0.027 KB total ukey len = 237.373170 GB avg = 0.015 KB total ukey num = 15.739583282 Billion total entry num = 11.280897230 Billion write speed all = 31.642064646 MB/sec (with seq num) write speed all = 30.525873383 MB/sec (without seq num) 2024/11/24-07:22:45.051003 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296201: 922119 keys, 37374600 bytes, temperature: kWarm 2024/11/24-07:22:45.051135 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404165051043, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296201, "file_size": 37374600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310565389, "largest_seqno": 4494943584, "table_properties": {"data_size": 31303664, "index_size": 1754304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819332, "raw_average_key_size": 28, "raw_value_size": 108398494, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922119, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404161, "oldest_key_time": 0, "file_creation_time": 1732404161, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:22:45.088373 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst) /dev/shm/ToplingTemp/Topling-296981-WnS1xx 2024/11/24-07:22:45.998863 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst): first pass time = 0.91's, 147.419'MB/sec 2024/11/24-07:22:47.356511 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst): old prealloc_size = 147639500, real_size = 32778816 2024/11/24-07:22:47.357133 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928075 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.62 avg-zval = 33.15 2024/11/24-07:22:47.357933 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:22:48.204709 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296202.sst): second pass time = 1.36's, 79.766'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.482'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2979.541'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 289.219'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30588.509'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.766'MB/sec zip pipeline throughput = 79.766'MB/sec entries = 928075 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.62 avg-zval = 33.15 usrkeys = 928075 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.62 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928075 bytes = 3248272 } seq expand size = 6496525 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4441 value = 3.5175 dict = 1.00 all = 3.6393 } Zip/UnZip{ index = 0.0957 value = 0.2843 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 2251.555155 GB avg = 0.200 KB (by entry num) total key len = 306.934768 GB avg = 0.027 KB total ukey len = 237.391731 GB avg = 0.015 KB total ukey num = 15.740511357 Billion total entry num = 11.281825305 Billion write speed all = 31.642552655 MB/sec (with seq num) write speed all = 30.526313742 MB/sec (without seq num) 2024/11/24-07:22:48.256005 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296202: 928075 keys, 36880500 bytes, temperature: kWarm 2024/11/24-07:22:48.256075 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404168256040, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296202, "file_size": 36880500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4494943526, "table_properties": {"data_size": 30769392, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986100, "raw_average_key_size": 28, "raw_value_size": 108231750, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928075, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404165, "oldest_key_time": 0, "file_creation_time": 1732404165, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:22:48.292789 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst) /dev/shm/ToplingTemp/Topling-296981-CCa6mu 2024/11/24-07:22:49.276540 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst): first pass time = 0.98's, 136.439'MB/sec 2024/11/24-07:22:50.566721 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst): old prealloc_size = 147639500, real_size = 31408816 2024/11/24-07:22:50.567265 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-07:22:50.568045 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:22:51.382700 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296203.sst): second pass time = 1.29's, 84.161'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.489'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3397.645'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 277.753'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34840.730'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.161'MB/sec zip pipeline throughput = 84.161'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4424 value = 3.6874 dict = 1.00 all = 3.7823 } Zip/UnZip{ index = 0.0958 value = 0.2712 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2251.670067 GB avg = 0.200 KB (by entry num) total key len = 306.960508 GB avg = 0.027 KB total ukey len = 237.410117 GB avg = 0.015 KB total ukey num = 15.741430657 Billion total entry num = 11.282744605 Billion write speed all = 31.643048493 MB/sec (with seq num) write speed all = 30.526762498 MB/sec (without seq num) 2024/11/24-07:22:51.429003 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296203: 919300 keys, 35485554 bytes, temperature: kWarm 2024/11/24-07:22:51.429072 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404171429037, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296203, "file_size": 35485554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4301634876, "largest_seqno": 4453480441, "table_properties": {"data_size": 29418080, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404168, "oldest_key_time": 0, "file_creation_time": 1732404168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:22:51.467768 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst) /dev/shm/ToplingTemp/Topling-296981-F3VDGw 2024/11/24-07:22:52.484695 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst): first pass time = 1.02's, 131.988'MB/sec 2024/11/24-07:22:53.859643 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst): old prealloc_size = 147639500, real_size = 33140000 2024/11/24-07:22:53.860203 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928129 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.61 avg-zval = 33.55 2024/11/24-07:22:53.861036 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:22:54.720320 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296204.sst): second pass time = 1.37's, 78.760'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.128'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 288.763'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30595.659'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.760'MB/sec zip pipeline throughput = 78.760'MB/sec entries = 928129 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 33.55 usrkeys = 928129 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928129 bytes = 3248464 } seq expand size = 6496903 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5159 value = 3.4753 dict = 1.00 all = 3.6039 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 2251.784794 GB avg = 0.200 KB (by entry num) total key len = 306.986496 GB avg = 0.027 KB total ukey len = 237.428680 GB avg = 0.015 KB total ukey num = 15.742358786 Billion total entry num = 11.283672734 Billion write speed all = 31.643482591 MB/sec (with seq num) write speed all = 30.527150848 MB/sec (without seq num) 2024/11/24-07:22:54.769068 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296204: 928129 keys, 37242038 bytes, temperature: kWarm 2024/11/24-07:22:54.769140 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404174769102, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296204, "file_size": 37242038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4494942201, "table_properties": {"data_size": 31142608, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987612, "raw_average_key_size": 28, "raw_value_size": 108230130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928129, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404171, "oldest_key_time": 0, "file_creation_time": 1732404171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:22:54.815648 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst) /dev/shm/ToplingTemp/Topling-296981-9Oja2v 2024/11/24-07:22:55.801307 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst): first pass time = 0.99's, 136.176'MB/sec 2024/11/24-07:22:57.118119 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst): old prealloc_size = 147639500, real_size = 31396336 2024/11/24-07:22:57.118664 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-07:22:57.119431 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:22:57.936671 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296205.sst): second pass time = 1.32's, 82.427'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.338'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.255'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 275.985'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34453.512'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.427'MB/sec zip pipeline throughput = 82.427'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6890 dict = 1.00 all = 3.7837 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.899707 GB avg = 0.200 KB (by entry num) total key len = 307.012236 GB avg = 0.027 KB total ukey len = 237.447066 GB avg = 0.015 KB total ukey num = 15.743278086 Billion total entry num = 11.284592034 Billion write speed all = 31.643963343 MB/sec (with seq num) write speed all = 30.527585055 MB/sec (without seq num) 2024/11/24-07:22:57.981512 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296205: 919300 keys, 35473074 bytes, temperature: kWarm 2024/11/24-07:22:57.981589 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404177981552, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296205, "file_size": 35473074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4319979540, "largest_seqno": 4470978936, "table_properties": {"data_size": 29405920, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404174, "oldest_key_time": 0, "file_creation_time": 1732404174, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:22:58.020370 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst) /dev/shm/ToplingTemp/Topling-296981-OU4fCw 2024/11/24-07:22:58.937651 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst): first pass time = 0.92's, 146.328'MB/sec 2024/11/24-07:23:00.297352 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst): old prealloc_size = 147639500, real_size = 33412656 2024/11/24-07:23:00.297929 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928135 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.61 avg-zval = 33.85 2024/11/24-07:23:00.298727 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:01.160904 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296206.sst): second pass time = 1.36's, 79.644'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.444'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.132'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 291.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33947.756'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.644'MB/sec zip pipeline throughput = 79.644'MB/sec entries = 928135 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 33.85 usrkeys = 928135 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928135 bytes = 3248480 } seq expand size = 6496945 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5122 value = 3.4452 dict = 1.00 all = 3.5777 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 2252.014434 GB avg = 0.200 KB (by entry num) total key len = 307.038224 GB avg = 0.027 KB total ukey len = 237.465628 GB avg = 0.015 KB total ukey num = 15.744206221 Billion total entry num = 11.285520169 Billion write speed all = 31.644441737 MB/sec (with seq num) write speed all = 30.528016143 MB/sec (without seq num) 2024/11/24-07:23:01.208881 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296206: 928135 keys, 37514906 bytes, temperature: kWarm 2024/11/24-07:23:01.208952 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404181208916, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296206, "file_size": 37514906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4494943738, "table_properties": {"data_size": 31414624, "index_size": 1765824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987780, "raw_average_key_size": 28, "raw_value_size": 108230050, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928135, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404178, "oldest_key_time": 0, "file_creation_time": 1732404178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296206, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:01.245212 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst) /dev/shm/ToplingTemp/Topling-296981-XygTTt 2024/11/24-07:23:02.127106 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst): first pass time = 0.88's, 152.199'MB/sec 2024/11/24-07:23:03.495806 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst): old prealloc_size = 147639500, real_size = 31429056 2024/11/24-07:23:03.496399 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-07:23:03.497178 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:04.316444 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296207.sst): second pass time = 1.37's, 79.305'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.751'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 276.462'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33815.683'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.305'MB/sec zip pipeline throughput = 79.305'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6849 dict = 1.00 all = 3.7802 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2645 } ---------------------------- total value len = 2252.129346 GB avg = 0.200 KB (by entry num) total key len = 307.063965 GB avg = 0.027 KB total ukey len = 237.484014 GB avg = 0.015 KB total ukey num = 15.745125521 Billion total entry num = 11.286439469 Billion write speed all = 31.644946209 MB/sec (with seq num) write speed all = 30.528473238 MB/sec (without seq num) 2024/11/24-07:23:04.360490 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296207: 919300 keys, 35505778 bytes, temperature: kWarm 2024/11/24-07:23:04.360562 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404184360526, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296207, "file_size": 35505778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4338187703, "largest_seqno": 4488621821, "table_properties": {"data_size": 29438640, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404181, "oldest_key_time": 0, "file_creation_time": 1732404181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296207, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:04.395825 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst) /dev/shm/ToplingTemp/Topling-296981-ap1hqt 2024/11/24-07:23:05.250855 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst): first pass time = 0.85's, 156.980'MB/sec 2024/11/24-07:23:06.598519 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst): old prealloc_size = 147639500, real_size = 33154240 2024/11/24-07:23:06.599070 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928435 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.56 avg-zval = 33.56 2024/11/24-07:23:06.599849 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:07.456929 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296208.sst): second pass time = 1.35's, 80.350'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.403'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.148'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 292.113'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31142.280'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.350'MB/sec zip pipeline throughput = 80.350'MB/sec entries = 928435 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 33.56 usrkeys = 928435 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928435 bytes = 3249536 } seq expand size = 6499045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5137 value = 3.4736 dict = 1.00 all = 3.6025 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2252.244067 GB avg = 0.200 KB (by entry num) total key len = 307.089961 GB avg = 0.027 KB total ukey len = 237.502583 GB avg = 0.015 KB total ukey num = 15.746053956 Billion total entry num = 11.287367904 Billion write speed all = 31.645457320 MB/sec (with seq num) write speed all = 30.528935864 MB/sec (without seq num) 2024/11/24-07:23:07.503219 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296208: 928435 keys, 37256850 bytes, temperature: kWarm 2024/11/24-07:23:07.503290 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404187503254, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296208, "file_size": 37256850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4494942352, "table_properties": {"data_size": 31155824, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996180, "raw_average_key_size": 28, "raw_value_size": 108221610, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404184, "oldest_key_time": 0, "file_creation_time": 1732404184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296208, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:07.540084 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst) /dev/shm/ToplingTemp/Topling-296981-fzlE1w 2024/11/24-07:23:08.536044 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst): first pass time = 1.00's, 134.767'MB/sec 2024/11/24-07:23:09.868858 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst): old prealloc_size = 147639500, real_size = 31243792 2024/11/24-07:23:09.869409 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920914 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.74 avg-zval = 31.76 2024/11/24-07:23:09.870199 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:10.685254 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296209.sst): second pass time = 1.33's, 81.401'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.271'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3337.552'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 275.379'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 29256.147'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.401'MB/sec zip pipeline throughput = 81.401'MB/sec entries = 920914 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.74 avg-zval = 31.76 usrkeys = 920914 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.74 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920914 bytes = 3223216 } seq expand size = 6446398 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4448 value = 3.7071 dict = 1.00 all = 3.7995 } Zip/UnZip{ index = 0.0957 value = 0.2698 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2252.358946 GB avg = 0.200 KB (by entry num) total key len = 307.115746 GB avg = 0.027 KB total ukey len = 237.521001 GB avg = 0.015 KB total ukey num = 15.746974870 Billion total entry num = 11.288288818 Billion write speed all = 31.645933374 MB/sec (with seq num) write speed all = 30.529365394 MB/sec (without seq num) 2024/11/24-07:23:10.729855 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296209: 920914 keys, 35324748 bytes, temperature: kWarm 2024/11/24-07:23:10.729925 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404190729889, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296209, "file_size": 35324748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4354401068, "largest_seqno": 4494941579, "table_properties": {"data_size": 29249984, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25785592, "raw_average_key_size": 28, "raw_value_size": 108432244, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920914, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404187, "oldest_key_time": 0, "file_creation_time": 1732404187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296209, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:10.765619 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst) /dev/shm/ToplingTemp/Topling-296981-u2nFvv 2024/11/24-07:23:11.752248 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst): first pass time = 0.99's, 136.042'MB/sec 2024/11/24-07:23:13.102610 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst): old prealloc_size = 147639500, real_size = 33159216 2024/11/24-07:23:13.103178 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926436 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.88 avg-zval = 33.64 2024/11/24-07:23:13.103954 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:13.963263 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296210.sst): second pass time = 1.35's, 80.230'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.671'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 288.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30640.645'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.230'MB/sec zip pipeline throughput = 80.230'MB/sec entries = 926436 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 33.64 usrkeys = 926436 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 33.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926436 bytes = 3242544 } seq expand size = 6485052 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5135 value = 3.4743 dict = 1.00 all = 3.6026 } Zip/UnZip{ index = 0.0951 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2252.473708 GB avg = 0.200 KB (by entry num) total key len = 307.141686 GB avg = 0.027 KB total ukey len = 237.539530 GB avg = 0.015 KB total ukey num = 15.747901306 Billion total entry num = 11.289215254 Billion write speed all = 31.646390426 MB/sec (with seq num) write speed all = 30.529776065 MB/sec (without seq num) 2024/11/24-07:23:14.009108 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296210: 926436 keys, 37256096 bytes, temperature: kWarm 2024/11/24-07:23:14.009180 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404194009144, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296210, "file_size": 37256096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4494943665, "table_properties": {"data_size": 31165024, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940208, "raw_average_key_size": 28, "raw_value_size": 108277536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404190, "oldest_key_time": 0, "file_creation_time": 1732404190, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296210, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:14.048964 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst) /dev/shm/ToplingTemp/Topling-296981-yZegZt 2024/11/24-07:23:14.993059 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst): first pass time = 0.94's, 142.174'MB/sec 2024/11/24-07:23:16.302153 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst): old prealloc_size = 147639500, real_size = 30925648 2024/11/24-07:23:16.302677 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924758 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.14 avg-zval = 31.28 2024/11/24-07:23:16.303432 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:17.116476 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296211.sst): second pass time = 1.31's, 82.839'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3525.743'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 273.326'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27432.935'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.839'MB/sec zip pipeline throughput = 82.839'MB/sec entries = 924758 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.14 avg-zval = 31.28 usrkeys = 924758 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.14 avg-zval = 31.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924758 bytes = 3121072 } seq expand size = 6473306 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4474 value = 3.7451 dict = 1.00 all = 3.8457 } Zip/UnZip{ index = 0.0957 value = 0.2670 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 2252.588506 GB avg = 0.200 KB (by entry num) total key len = 307.167580 GB avg = 0.027 KB total ukey len = 237.558025 GB avg = 0.015 KB total ukey num = 15.748826064 Billion total entry num = 11.290140012 Billion write speed all = 31.646896122 MB/sec (with seq num) write speed all = 30.530233827 MB/sec (without seq num) 2024/11/24-07:23:17.159910 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296211: 924758 keys, 34900796 bytes, temperature: kWarm 2024/11/24-07:23:17.159994 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404197159945, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296211, "file_size": 34900796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4377304925, "largest_seqno": 4494941593, "table_properties": {"data_size": 28923968, "index_size": 1770304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893224, "raw_average_key_size": 28, "raw_value_size": 108324576, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924758, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404194, "oldest_key_time": 0, "file_creation_time": 1732404194, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296211, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:17.195126 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst) /dev/shm/ToplingTemp/Topling-296981-M8kY7u 2024/11/24-07:23:18.043796 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst): first pass time = 0.85's, 158.157'MB/sec 2024/11/24-07:23:19.383687 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst): old prealloc_size = 147639500, real_size = 32622112 2024/11/24-07:23:19.384227 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922766 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0306 GB avg-val = 117.45 avg-zval = 33.20 2024/11/24-07:23:19.385011 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:20.212353 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296212.sst): second pass time = 1.34's, 80.937'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.689'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 288.603'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33531.634'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.937'MB/sec zip pipeline throughput = 80.937'MB/sec entries = 922766 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 33.20 usrkeys = 922766 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.45 avg-zval = 33.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922766 bytes = 3229696 } seq expand size = 6459362 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5108 value = 3.5377 dict = 1.00 all = 3.6562 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 2252.703346 GB avg = 0.200 KB (by entry num) total key len = 307.193417 GB avg = 0.027 KB total ukey len = 237.576481 GB avg = 0.015 KB total ukey num = 15.749748830 Billion total entry num = 11.291062778 Billion write speed all = 31.647424042 MB/sec (with seq num) write speed all = 30.530713222 MB/sec (without seq num) 2024/11/24-07:23:20.260144 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296212: 922766 keys, 36709730 bytes, temperature: kWarm 2024/11/24-07:23:20.260215 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404200260178, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296212, "file_size": 36709730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4494943460, "table_properties": {"data_size": 30635408, "index_size": 1755840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25837448, "raw_average_key_size": 28, "raw_value_size": 108380408, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922766, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404197, "oldest_key_time": 0, "file_creation_time": 1732404197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296212, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:20.298759 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst) /dev/shm/ToplingTemp/Topling-296981-1EZxMu 2024/11/24-07:23:21.287113 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst): first pass time = 0.99's, 135.806'MB/sec 2024/11/24-07:23:22.606897 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst): old prealloc_size = 147639500, real_size = 30903600 2024/11/24-07:23:22.607435 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927862 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0289 GB avg-val = 116.65 avg-zval = 31.15 2024/11/24-07:23:22.608226 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:23.430822 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296213.sst): second pass time = 1.32's, 82.089'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.487'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.667'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 270.420'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30613.980'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.089'MB/sec zip pipeline throughput = 82.089'MB/sec entries = 927862 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 31.15 usrkeys = 927862 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 31.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927862 bytes = 3247536 } seq expand size = 6495034 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.5152 value = 3.7444 dict = 1.00 all = 3.8342 } Zip/UnZip{ index = 0.0951 value = 0.2671 dict = 1.00 all = 0.2608 } ---------------------------- total value len = 2252.818079 GB avg = 0.200 KB (by entry num) total key len = 307.219397 GB avg = 0.027 KB total ukey len = 237.595038 GB avg = 0.015 KB total ukey num = 15.750676692 Billion total entry num = 11.291990640 Billion write speed all = 31.647904397 MB/sec (with seq num) write speed all = 30.531146245 MB/sec (without seq num) 2024/11/24-07:23:23.475063 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296213: 927862 keys, 35005008 bytes, temperature: kWarm 2024/11/24-07:23:23.475137 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404203475101, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296213, "file_size": 35005008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4494941606, "table_properties": {"data_size": 28906656, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980136, "raw_average_key_size": 28, "raw_value_size": 108237640, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927862, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404200, "oldest_key_time": 0, "file_creation_time": 1732404200, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296213, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:23.513752 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst) /dev/shm/ToplingTemp/Topling-296981-R0iXdw 2024/11/24-07:23:24.327552 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst): first pass time = 0.81's, 164.935'MB/sec 2024/11/24-07:23:25.615628 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst): old prealloc_size = 147639500, real_size = 31470592 2024/11/24-07:23:25.616162 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.07 2024/11/24-07:23:25.616933 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:26.431628 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296214.sst): second pass time = 1.29's, 84.266'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.564'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.640'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 276.890'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 28233.411'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.266'MB/sec zip pipeline throughput = 84.266'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.07 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4455 value = 3.6796 dict = 1.00 all = 3.7758 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2252.932991 GB avg = 0.199 KB (by entry num) total key len = 307.245138 GB avg = 0.027 KB total ukey len = 237.613424 GB avg = 0.015 KB total ukey num = 15.751595992 Billion total entry num = 11.292909940 Billion write speed all = 31.648469129 MB/sec (with seq num) write speed all = 30.531661489 MB/sec (without seq num) 2024/11/24-07:23:26.476138 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296214: 919300 keys, 35547314 bytes, temperature: kWarm 2024/11/24-07:23:26.476211 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404206476174, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296214, "file_size": 35547314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269349065, "largest_seqno": 4422041801, "table_properties": {"data_size": 29480368, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404203, "oldest_key_time": 0, "file_creation_time": 1732404203, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296214, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:26.515550 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst) /dev/shm/ToplingTemp/Topling-296981-xa8szt 2024/11/24-07:23:27.533121 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst): first pass time = 1.02's, 131.905'MB/sec 2024/11/24-07:23:28.877593 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst): old prealloc_size = 147639500, real_size = 32418448 2024/11/24-07:23:28.878152 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928246 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.59 avg-zval = 32.76 2024/11/24-07:23:28.878914 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:29.710640 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296215.sst): second pass time = 1.34's, 80.571'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.741'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 287.335'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34903.766'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.571'MB/sec zip pipeline throughput = 80.571'MB/sec entries = 928246 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 32.76 usrkeys = 928246 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 32.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928246 bytes = 3248880 } seq expand size = 6497722 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4442 value = 3.5591 dict = 1.00 all = 3.6748 } Zip/UnZip{ index = 0.0957 value = 0.2810 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 2253.047716 GB avg = 0.199 KB (by entry num) total key len = 307.271129 GB avg = 0.027 KB total ukey len = 237.631989 GB avg = 0.015 KB total ukey num = 15.752524238 Billion total entry num = 11.293838186 Billion write speed all = 31.648925754 MB/sec (with seq num) write speed all = 30.532071587 MB/sec (without seq num) 2024/11/24-07:23:29.755595 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296215: 928246 keys, 36524308 bytes, temperature: kWarm 2024/11/24-07:23:29.755667 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404209755630, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296215, "file_size": 36524308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4494943050, "table_properties": {"data_size": 30408640, "index_size": 1777536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990888, "raw_average_key_size": 28, "raw_value_size": 108226876, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928246, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404206, "oldest_key_time": 0, "file_creation_time": 1732404206, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296215, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:29.793230 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst) /dev/shm/ToplingTemp/Topling-296981-hkx8vu 2024/11/24-07:23:30.763921 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst): first pass time = 0.97's, 138.276'MB/sec 2024/11/24-07:23:32.087887 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst): old prealloc_size = 147639500, real_size = 30816352 2024/11/24-07:23:32.088459 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926301 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.90 avg-zval = 31.10 2024/11/24-07:23:32.089272 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:32.909075 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296216.sst): second pass time = 1.32's, 81.835'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.514'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.280'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 266.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35599.802'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.835'MB/sec zip pipeline throughput = 81.835'MB/sec entries = 926301 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.90 avg-zval = 31.10 usrkeys = 926301 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.90 avg-zval = 31.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926301 bytes = 3242064 } seq expand size = 6484107 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4449 value = 3.7584 dict = 1.00 all = 3.8445 } Zip/UnZip{ index = 0.0957 value = 0.2661 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 2253.162481 GB avg = 0.199 KB (by entry num) total key len = 307.297065 GB avg = 0.027 KB total ukey len = 237.650515 GB avg = 0.015 KB total ukey num = 15.753450539 Billion total entry num = 11.294764487 Billion write speed all = 31.649413698 MB/sec (with seq num) write speed all = 30.532512087 MB/sec (without seq num) 2024/11/24-07:23:32.952056 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296216: 926301 keys, 34911254 bytes, temperature: kWarm 2024/11/24-07:23:32.952123 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404212952091, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296216, "file_size": 34911254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4494939950, "table_properties": {"data_size": 28810896, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25936428, "raw_average_key_size": 28, "raw_value_size": 108281434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926301, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404209, "oldest_key_time": 0, "file_creation_time": 1732404209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296216, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:32.987565 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst) /dev/shm/ToplingTemp/Topling-296981-TJizmx 2024/11/24-07:23:33.869005 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst): first pass time = 0.88's, 152.278'MB/sec 2024/11/24-07:23:35.113161 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst): old prealloc_size = 147639500, real_size = 27693728 2024/11/24-07:23:35.113723 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1078 GB zip-val = 0.0256 GB avg-val = 114.00 avg-zval = 27.12 2024/11/24-07:23:35.114492 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:35.949964 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296217.sst): second pass time = 1.24's, 86.660'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.913'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.751'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 244.713'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 34637.754'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 86.660'MB/sec zip pipeline throughput = 86.660'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 27.12 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 27.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0256 GB dict = 1.07 MB all = 0.0317 GB } UnZip/Zip{ index = 10.3775 value = 4.2032 dict = 1.00 all = 4.2307 } Zip/UnZip{ index = 0.0964 value = 0.2379 dict = 1.00 all = 0.2364 } ---------------------------- total value len = 2253.276850 GB avg = 0.199 KB (by entry num) total key len = 307.323531 GB avg = 0.027 KB total ukey len = 237.669419 GB avg = 0.015 KB total ukey num = 15.754395735 Billion total entry num = 11.295709683 Billion write speed all = 31.649964871 MB/sec (with seq num) write speed all = 30.533011777 MB/sec (without seq num) 2024/11/24-07:23:35.989788 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296217: 945196 keys, 31724912 bytes, temperature: kWarm 2024/11/24-07:23:35.989858 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404215989823, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296217, "file_size": 31724912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4292133110, "largest_seqno": 4393367055, "table_properties": {"data_size": 25635600, "index_size": 1821632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404212, "oldest_key_time": 0, "file_creation_time": 1732404212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296217, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:36.026397 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst) /dev/shm/ToplingTemp/Topling-296981-oxbHyt 2024/11/24-07:23:36.899729 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst): first pass time = 0.87's, 153.691'MB/sec 2024/11/24-07:23:38.149452 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst): old prealloc_size = 147639500, real_size = 28296688 2024/11/24-07:23:38.150043 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944456 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0263 GB avg-val = 114.11 avg-zval = 27.81 2024/11/24-07:23:38.150830 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:38.991258 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296218.sst): second pass time = 1.25's, 86.293'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.459'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 249.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 32996.527'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.293'MB/sec zip pipeline throughput = 86.293'MB/sec entries = 944456 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.11 avg-zval = 27.81 usrkeys = 944456 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.11 avg-zval = 27.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944456 bytes = 3305616 } seq expand size = 6611192 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0263 GB dict = 1.07 MB all = 0.0324 GB } UnZip/Zip{ index = 10.5134 value = 4.1035 dict = 1.00 all = 4.1367 } Zip/UnZip{ index = 0.0951 value = 0.2437 dict = 1.00 all = 0.2417 } ---------------------------- total value len = 2253.391234 GB avg = 0.199 KB (by entry num) total key len = 307.349975 GB avg = 0.027 KB total ukey len = 237.688308 GB avg = 0.015 KB total ukey num = 15.755340191 Billion total entry num = 11.296654139 Billion write speed all = 31.650515778 MB/sec (with seq num) write speed all = 30.533511283 MB/sec (without seq num) 2024/11/24-07:23:39.032309 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296218: 944456 keys, 32445654 bytes, temperature: kWarm 2024/11/24-07:23:39.032380 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404219032343, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296218, "file_size": 32445654, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4494943710, "table_properties": {"data_size": 26263712, "index_size": 1796672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26444768, "raw_average_key_size": 28, "raw_value_size": 107773016, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944456, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404216, "oldest_key_time": 0, "file_creation_time": 1732404216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296218, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:39.072456 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst) /dev/shm/ToplingTemp/Topling-296981-CjTWLw 2024/11/24-07:23:39.942475 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst): first pass time = 0.87's, 154.276'MB/sec 2024/11/24-07:23:41.226158 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst): old prealloc_size = 147639500, real_size = 31307424 2024/11/24-07:23:41.226689 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.89 2024/11/24-07:23:41.227455 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:42.037636 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296219.sst): second pass time = 1.28's, 84.584'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3450.817'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 274.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31612.668'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.584'MB/sec zip pipeline throughput = 84.584'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.89 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4439 value = 3.7002 dict = 1.00 all = 3.7932 } Zip/UnZip{ index = 0.0957 value = 0.2703 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2253.506147 GB avg = 0.199 KB (by entry num) total key len = 307.375716 GB avg = 0.027 KB total ukey len = 237.706694 GB avg = 0.015 KB total ukey num = 15.756259491 Billion total entry num = 11.297573439 Billion write speed all = 31.651062479 MB/sec (with seq num) write speed all = 30.534009145 MB/sec (without seq num) 2024/11/24-07:23:42.081157 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296219: 919300 keys, 35384114 bytes, temperature: kWarm 2024/11/24-07:23:42.081225 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404222081190, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296219, "file_size": 35384114, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269300222, "largest_seqno": 4413092809, "table_properties": {"data_size": 29316944, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404219, "oldest_key_time": 0, "file_creation_time": 1732404219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296219, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:42.125312 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst) /dev/shm/ToplingTemp/Topling-296981-vXtsTw 2024/11/24-07:23:43.085620 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst): first pass time = 0.96's, 139.771'MB/sec 2024/11/24-07:23:44.376612 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst): old prealloc_size = 147639500, real_size = 30078016 2024/11/24-07:23:44.377199 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931799 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0281 GB avg-val = 116.04 avg-zval = 30.11 2024/11/24-07:23:44.377985 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:45.201159 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296220.sst): second pass time = 1.29's, 83.829'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.465'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.069'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 262.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32883.839'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.829'MB/sec zip pipeline throughput = 83.829'MB/sec entries = 931799 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.04 avg-zval = 30.11 usrkeys = 931799 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.04 avg-zval = 30.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931799 bytes = 3261312 } seq expand size = 6522593 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0342 GB } UnZip/Zip{ index = 10.4428 value = 3.8534 dict = 1.00 all = 3.9258 } Zip/UnZip{ index = 0.0958 value = 0.2595 dict = 1.00 all = 0.2547 } ---------------------------- total value len = 2253.620797 GB avg = 0.199 KB (by entry num) total key len = 307.401806 GB avg = 0.027 KB total ukey len = 237.725330 GB avg = 0.015 KB total ukey num = 15.757191290 Billion total entry num = 11.298505238 Billion write speed all = 31.651564394 MB/sec (with seq num) write speed all = 30.534462606 MB/sec (without seq num) 2024/11/24-07:23:45.245186 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296220: 931799 keys, 34188414 bytes, temperature: kWarm 2024/11/24-07:23:45.245257 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404225245220, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296220, "file_size": 34188414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4357999832, "largest_seqno": 4494940952, "table_properties": {"data_size": 28060336, "index_size": 1784576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090372, "raw_average_key_size": 28, "raw_value_size": 108127358, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931799, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404222, "oldest_key_time": 0, "file_creation_time": 1732404222, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296220, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:45.284257 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst) /dev/shm/ToplingTemp/Topling-296981-KrLkcu 2024/11/24-07:23:46.217668 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst): first pass time = 0.93's, 143.799'MB/sec 2024/11/24-07:23:47.496742 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst): old prealloc_size = 147639500, real_size = 30113536 2024/11/24-07:23:47.497334 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 941036 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0281 GB avg-val = 114.63 avg-zval = 29.84 2024/11/24-07:23:47.498120 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:48.345907 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296221.sst): second pass time = 1.28's, 84.389'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.968'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.328'MB/sec write SST data time = 0.13's, 262.909'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 33580.462'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.389'MB/sec zip pipeline throughput = 84.389'MB/sec entries = 941036 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.63 avg-zval = 29.84 usrkeys = 941036 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.63 avg-zval = 29.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 941036 bytes = 3293648 } seq expand size = 6587252 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.4459 value = 3.8420 dict = 1.00 all = 3.9183 } Zip/UnZip{ index = 0.0957 value = 0.2603 dict = 1.00 all = 0.2552 } ---------------------------- total value len = 2253.735253 GB avg = 0.199 KB (by entry num) total key len = 307.428155 GB avg = 0.027 KB total ukey len = 237.744151 GB avg = 0.015 KB total ukey num = 15.758132326 Billion total entry num = 11.299446274 Billion write speed all = 31.652074415 MB/sec (with seq num) write speed all = 30.534923004 MB/sec (without seq num) 2024/11/24-07:23:48.388927 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296221: 941036 keys, 34253726 bytes, temperature: kWarm 2024/11/24-07:23:48.389025 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404228388962, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296221, "file_size": 34253726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4494943277, "table_properties": {"data_size": 28076336, "index_size": 1801728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26349008, "raw_average_key_size": 28, "raw_value_size": 107868824, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 941036, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404225, "oldest_key_time": 0, "file_creation_time": 1732404225, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296221, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:48.425932 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst) /dev/shm/ToplingTemp/Topling-296981-HSyeBt 2024/11/24-07:23:49.417388 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst): first pass time = 0.99's, 135.380'MB/sec 2024/11/24-07:23:50.710584 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst): old prealloc_size = 147639500, real_size = 31377856 2024/11/24-07:23:50.711143 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-07:23:50.711919 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:51.528493 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296222.sst): second pass time = 1.29's, 83.935'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.209'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.861'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 271.246'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36480.734'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.935'MB/sec zip pipeline throughput = 83.935'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3783 value = 3.6927 dict = 1.00 all = 3.7979 } Zip/UnZip{ index = 0.0964 value = 0.2708 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2253.850165 GB avg = 0.199 KB (by entry num) total key len = 307.453895 GB avg = 0.027 KB total ukey len = 237.762537 GB avg = 0.015 KB total ukey num = 15.759051626 Billion total entry num = 11.300365574 Billion write speed all = 31.652567713 MB/sec (with seq num) write speed all = 30.535369355 MB/sec (without seq num) 2024/11/24-07:23:51.572570 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296222: 919300 keys, 35339698 bytes, temperature: kWarm 2024/11/24-07:23:51.572638 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404231572604, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296222, "file_size": 35339698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4283157599, "largest_seqno": 4400587726, "table_properties": {"data_size": 29376240, "index_size": 1771584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404228, "oldest_key_time": 0, "file_creation_time": 1732404228, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296222, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:51.609841 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst) /dev/shm/ToplingTemp/Topling-296981-x6Dl3x 2024/11/24-07:23:52.582184 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst): first pass time = 0.97's, 138.042'MB/sec 2024/11/24-07:23:53.885431 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst): old prealloc_size = 147639500, real_size = 30792832 2024/11/24-07:23:53.886040 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925464 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 117.03 avg-zval = 31.11 2024/11/24-07:23:53.886821 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:54.703749 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296223.sst): second pass time = 1.30's, 83.155'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.491'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3030.384'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 263.883'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33346.722'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.155'MB/sec zip pipeline throughput = 83.155'MB/sec entries = 925464 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.03 avg-zval = 31.11 usrkeys = 925464 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.03 avg-zval = 31.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925464 bytes = 3123456 } seq expand size = 6478248 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4418 value = 3.7621 dict = 1.00 all = 3.8602 } Zip/UnZip{ index = 0.0958 value = 0.2658 dict = 1.00 all = 0.2591 } ---------------------------- total value len = 2253.964948 GB avg = 0.199 KB (by entry num) total key len = 307.479808 GB avg = 0.027 KB total ukey len = 237.781046 GB avg = 0.015 KB total ukey num = 15.759977090 Billion total entry num = 11.301291038 Billion write speed all = 31.653064376 MB/sec (with seq num) write speed all = 30.535818363 MB/sec (without seq num) 2024/11/24-07:23:54.746999 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296223: 925464 keys, 34769846 bytes, temperature: kWarm 2024/11/24-07:23:54.747070 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404234747034, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296223, "file_size": 34769846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4400587727, "largest_seqno": 4494941935, "table_properties": {"data_size": 28788656, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912992, "raw_average_key_size": 28, "raw_value_size": 108304852, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925464, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404231, "oldest_key_time": 0, "file_creation_time": 1732404231, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296223, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:54.784482 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst) /dev/shm/ToplingTemp/Topling-296981-aN3RPw 2024/11/24-07:23:55.734387 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst): first pass time = 0.95's, 141.302'MB/sec 2024/11/24-07:23:57.067083 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst): old prealloc_size = 147639500, real_size = 32666624 2024/11/24-07:23:57.067665 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932207 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.98 avg-zval = 32.88 2024/11/24-07:23:57.068460 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:23:57.933538 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296224.sst): second pass time = 1.33's, 81.210'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.451'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.237'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 281.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32091.158'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.210'MB/sec zip pipeline throughput = 81.210'MB/sec entries = 932207 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.98 avg-zval = 32.88 usrkeys = 932207 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.98 avg-zval = 32.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932207 bytes = 3262736 } seq expand size = 6525449 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4455 value = 3.5276 dict = 1.00 all = 3.6492 } Zip/UnZip{ index = 0.0957 value = 0.2835 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 2254.079590 GB avg = 0.199 KB (by entry num) total key len = 307.505910 GB avg = 0.027 KB total ukey len = 237.799690 GB avg = 0.015 KB total ukey num = 15.760909297 Billion total entry num = 11.302223245 Billion write speed all = 31.653540252 MB/sec (with seq num) write speed all = 30.536246675 MB/sec (without seq num) 2024/11/24-07:23:57.979634 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296224: 932207 keys, 36780038 bytes, temperature: kWarm 2024/11/24-07:23:57.979705 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404237979668, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296224, "file_size": 36780038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4494943618, "table_properties": {"data_size": 30648496, "index_size": 1784896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101796, "raw_average_key_size": 28, "raw_value_size": 108115970, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932207, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404234, "oldest_key_time": 0, "file_creation_time": 1732404234, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296224, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:23:58.017894 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst) /dev/shm/ToplingTemp/Topling-296981-Jwzo9v 2024/11/24-07:23:58.994450 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst): first pass time = 0.98's, 137.446'MB/sec 2024/11/24-07:24:00.320074 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst): old prealloc_size = 147639500, real_size = 31229600 2024/11/24-07:24:00.320640 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0292 GB avg-val = 118.00 avg-zval = 31.81 2024/11/24-07:24:00.321382 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:24:01.131585 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296225.sst): second pass time = 1.32's, 81.888'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.367'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.398'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 271.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32326.600'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.888'MB/sec zip pipeline throughput = 81.888'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.81 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.09 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4447 value = 3.7100 dict = 1.00 all = 3.8139 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 2254.194502 GB avg = 0.199 KB (by entry num) total key len = 307.531651 GB avg = 0.027 KB total ukey len = 237.818076 GB avg = 0.015 KB total ukey num = 15.761828597 Billion total entry num = 11.303142545 Billion write speed all = 31.654027389 MB/sec (with seq num) write speed all = 30.536687088 MB/sec (without seq num) 2024/11/24-07:24:01.175943 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296225: 919300 keys, 35191410 bytes, temperature: kWarm 2024/11/24-07:24:01.176024 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404241175988, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296225, "file_size": 35191410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4319388752, "largest_seqno": 4431382771, "table_properties": {"data_size": 29239248, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404238, "oldest_key_time": 0, "file_creation_time": 1732404238, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296225, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:24:01.215683 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst) /dev/shm/ToplingTemp/Topling-296981-bMYgAx 2024/11/24-07:24:02.215700 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst): first pass time = 1.00's, 134.222'MB/sec 2024/11/24-07:24:03.527367 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst): old prealloc_size = 147639500, real_size = 30271504 2024/11/24-07:24:03.527958 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930828 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.19 avg-zval = 30.36 2024/11/24-07:24:03.528698 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:24:04.354749 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296226.sst): second pass time = 1.31's, 82.546'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.272'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.480'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.13's, 259.419'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31700.470'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.546'MB/sec zip pipeline throughput = 82.546'MB/sec entries = 930828 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 30.36 usrkeys = 930828 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.19 avg-zval = 30.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930828 bytes = 3025200 } seq expand size = 6515796 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0341 GB } UnZip/Zip{ index = 10.4443 value = 3.8277 dict = 1.00 all = 3.9307 } Zip/UnZip{ index = 0.0957 value = 0.2613 dict = 1.00 all = 0.2544 } ---------------------------- total value len = 2254.309173 GB avg = 0.199 KB (by entry num) total key len = 307.557714 GB avg = 0.027 KB total ukey len = 237.836693 GB avg = 0.015 KB total ukey num = 15.762759425 Billion total entry num = 11.304073373 Billion write speed all = 31.654505657 MB/sec (with seq num) write speed all = 30.537117844 MB/sec (without seq num) 2024/11/24-07:24:04.399223 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296226: 930828 keys, 34146370 bytes, temperature: kWarm 2024/11/24-07:24:04.399291 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404244399256, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296226, "file_size": 34146370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4431382772, "largest_seqno": 4494941947, "table_properties": {"data_size": 28256128, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063184, "raw_average_key_size": 28, "raw_value_size": 108154664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930828, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404241, "oldest_key_time": 0, "file_creation_time": 1732404241, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296226, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:24:04.441146 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst) /dev/shm/ToplingTemp/Topling-296981-ntlCxw 2024/11/24-07:24:04.636233 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst): first pass time = 0.20's, 252.448'MB/sec 2024/11/24-07:24:05.152855 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst): old prealloc_size = 147639500, real_size = 13711776 2024/11/24-07:24:05.153124 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 340227 prefix = 4 raw-key = 0.0054 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0397 GB zip-val = 0.0130 GB avg-val = 116.73 avg-zval = 38.15 2024/11/24-07:24:05.153457 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.049 GB 2024/11/24-07:24:05.461289 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00/296227.sst): second pass time = 0.52's, 76.984'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.31's, 49.370'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2635.120'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.326'MB/sec write SST data time = 0.05's, 321.605'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.406'MB, throughput = 14107.751'MB/sec zip my value time = 0.52's, unzip length = 0.040'GB zip my value throughput = 76.984'MB/sec zip pipeline throughput = 76.984'MB/sec entries = 340227 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 38.15 usrkeys = 340227 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.73 avg-zval = 38.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 340227 bytes = 1190816 } seq expand size = 2381589 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0068 GB value = 0.0397 GB dict = 0.41 MB all = 0.0492 GB } __ZipSize{ index = 0.0006 GB value = 0.0130 GB dict = 0.41 MB all = 0.0152 GB } UnZip/Zip{ index = 10.4977 value = 3.0599 dict = 1.00 all = 3.2338 } Zip/UnZip{ index = 0.0953 value = 0.3268 dict = 1.00 all = 0.3092 } ---------------------------- total value len = 2254.351267 GB avg = 0.199 KB (by entry num) total key len = 307.567240 GB avg = 0.027 KB total ukey len = 237.843497 GB avg = 0.015 KB total ukey num = 15.763099652 Billion total entry num = 11.304413600 Billion write speed all = 31.654710689 MB/sec (with seq num) write speed all = 30.537304524 MB/sec (without seq num) 2024/11/24-07:24:05.484717 139957198378560 [compaction_job.cc:1948] [default] [JOB 43370] Generated table #296227: 340227 keys, 15226247 bytes, temperature: kWarm 2024/11/24-07:24:05.484775 139957198378560 EVENT_LOG_v1 {"time_micros": 1732404245484743, "cf_name": "default", "job": 43370, "event": "table_file_creation", "file_number": 296227, "file_size": 15226247, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4494942386, "table_properties": {"data_size": 12978320, "index_size": 648192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9526356, "raw_average_key_size": 28, "raw_value_size": 39713006, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 340227, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732404244, "oldest_key_time": 0, "file_creation_time": 1732404244, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 296227, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:24:05.500654 139957198378560 [compaction_job.cc:703] job-43370: subcompact[0], size: 0.938363 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43370/att-00 [296201,296202,296203,296204,296205,296206,296207,296208,296209,296210,296211,296212,296213,296214,296215,296216,296217,296218,296219,296220,296221,296222,296223,296224,296225,296226,296227] 2024/11/24-07:24:05.658145 139957198378560 (Original Log Time 2024/11/24-07:22:41.381723) EVENT_LOG_v1 {"time_micros": 1732404161381683, "job": 43370, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [296140], "files_L3": [296181, 296072, 296073, 296074, 296075, 296076, 296077, 296078, 296079, 296080, 296081, 296082, 296083, 296084, 296085, 296087, 296088, 296089, 296090, 296091, 296092, 296093, 296094, 296095, 296096, 296097, 296182], "score": 0.984143, "input_data_size": 964032939, "oldest_snapshot_seqno": 4495256117} 2024/11/24-07:24:05.658149 139957198378560 (Original Log Time 2024/11/24-07:24:05.655340) [compaction_job.cc:2022] [default] [JOB 43370] Compacted 1@2 + 27@3 files to L3 => 938362983 bytes 2024/11/24-07:24:05.658150 139957198378560 (Original Log Time 2024/11/24-07:24:05.658072) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 27 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.2 wr, level 3, files in(1, 27) out(27 +0 blob) MB in(27.1, 892.3 +0.0 blob) out(894.9 +0.0 blob), read-write-amplify(67.0) write-amplify(33.0) OK, records in: 24535294, records dropped: 96805 output_compression: Snappy 2024/11/24-07:24:05.658153 139957198378560 (Original Log Time 2024/11/24-07:24:05.658119) EVENT_LOG_v1 {"time_micros": 1732404245658096, "cf": "default", "job": 43370, "event": "compaction_finished", "compaction_time_micros": 84118937, "compaction_time_cpu_micros": 36143992, "output_level": 3, "num_output_files": 27, "total_output_size": 938362983, "num_input_records": 24535294, "num_output_records": 24438489, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 637053949, "file_range_sync_nanos": 0, "file_fsync_nanos": 19069, "file_prepare_write_nanos": 74339, "lsm_state": [0, 0, 0, 27, 0, 0, 0]} 2024/11/24-07:24:05.785263 139957198378560 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732404161,"executesMs":84415,"compactionJobId":43370,"attempt":0,"compactionInputRawBytes":3552648584,"compactionInputZipBytes":964032939,"compactionOutputRawBytes":0,"compactionOutputZipBytes":938362983,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-84a63fadbe1fd8e7f21dab4ca9cee2d3-45850fb8fcb017d1-00"}}, response = 2024/11/24-07:24:05.785361 139957198378560 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43370/att-00: olev 3, work 84.416 s, result 6.019 ms, install 2.828 ms, input{raw 3.309 GiB zip 919.373 MiB}