2024/11/23-13:11:44.673335 139957483730496 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00: fork to child time = 0.011678 sec 2024/11/23-13:11:44.673972 139957483730496 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00 2024/11/23-13:11:44.682703 139957483730496 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00 2024/11/23-13:11:44.683621 139957483730496 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-13:11:44.683872 139957483730496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-13:11:44.737542 139957483730496 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26600/att-00/MANIFEST-206347 2024/11/23-13:11:44.775589 139957483730496 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26600/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 260760, last_sequence is 3990318035, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 260570 2024/11/23-13:11:44.775602 139957483730496 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-13:11:44.775722 139957483730496 [version_set.cc:5721] Creating manifest 260760 2024/11/23-13:11:44.816438 139957483730496 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-13:11:44.842037 139957483730496 [compaction_job.cc:2437] [default] [JOB 26600] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-13:11:44.842068 139957483730496 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 19926 Base level 2, inputs: [260715(27MB)], [260676(34MB) 260677(33MB) 260678(34MB) 260679(34MB) 260680(36MB) 260681(26MB) 260638(36MB) 260639(34MB) 260640(34MB) 260641(34MB) 260642(34MB) 260643(35MB) 260644(35MB) 260645(35MB) 260721(5411KB)] 2024/11/23-13:11:44.848835 139957483730496 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:11:44.848865 139957483730496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:11:44.848885 139957483730496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-13:11:44.848890 139957483730496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:11:44.848902 139957483730496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-13:11:45.415475 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst) /dev/shm/ToplingTemp/Topling-56986-sIeb7e 2024/11/23-13:11:45.965322 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst): first pass time = 0.55's, 244.121'MB/sec 2024/11/23-13:11:48.528954 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst): old prealloc_size = 147639500, real_size = 31822240 2024/11/23-13:11:48.529496 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931576 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0298 GB avg-val = 116.08 avg-zval = 31.99 2024/11/23-13:11:48.530341 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:11:49.371404 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260761.sst): second pass time = 2.56's, 42.194'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.931'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3475.506'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 274.624'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36613.289'MB/sec zip my value time = 2.56's, unzip length = 0.108'GB zip my value throughput = 42.194'MB/sec zip pipeline throughput = 106.146'MB/sec entries = 931576 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.08 avg-zval = 31.99 usrkeys = 931576 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.08 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931576 bytes = 3493424 } seq expand size = 6521032 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4433 value = 3.6280 dict = 1.00 all = 3.7111 } Zip/UnZip{ index = 0.0958 value = 0.2756 dict = 1.00 all = 0.2695 } ---------------------------- total value len = 8.963274 GB avg = 0.168 KB (by entry num) total key len = 1.466905 GB avg = 0.028 KB total ukey len = 1.016510 GB avg = 0.015 KB total ukey num = 0.066482011 Billion total entry num = 0.053289740 Billion write speed all = 0.677409462 MB/sec (with seq num) write speed all = 0.649721369 MB/sec (without seq num) 2024/11/23-13:11:49.415956 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260761: 931576 keys, 36166414 bytes, temperature: kWarm 2024/11/23-13:11:49.416068 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338709416004, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260761, "file_size": 36166414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3990169303, "table_properties": {"data_size": 29805072, "index_size": 1784064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26084128, "raw_average_key_size": 28, "raw_value_size": 108133648, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931576, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338705, "oldest_key_time": 0, "file_creation_time": 1732338705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260761, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:11:49.454170 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst) /dev/shm/ToplingTemp/Topling-56986-gIy6ve 2024/11/23-13:11:50.299957 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst): first pass time = 0.85's, 158.697'MB/sec 2024/11/23-13:11:52.809390 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst): old prealloc_size = 147639500, real_size = 30974704 2024/11/23-13:11:52.809918 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924995 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.10 avg-zval = 31.32 2024/11/23-13:11:52.810760 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:11:53.637488 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260762.sst): second pass time = 2.51's, 43.178'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.626'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3508.396'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 271.825'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36001.665'MB/sec zip my value time = 2.51's, unzip length = 0.108'GB zip my value throughput = 43.178'MB/sec zip pipeline throughput = 113.055'MB/sec entries = 924995 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.10 avg-zval = 31.32 usrkeys = 924995 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.10 avg-zval = 31.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924995 bytes = 3468752 } seq expand size = 6474965 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4433 value = 3.7387 dict = 1.00 all = 3.8025 } Zip/UnZip{ index = 0.0958 value = 0.2675 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 9.078067 GB avg = 0.167 KB (by entry num) total key len = 1.492805 GB avg = 0.028 KB total ukey len = 1.035009 GB avg = 0.015 KB total ukey num = 0.067407006 Billion total entry num = 0.054214735 Billion write speed all = 0.686356879 MB/sec (with seq num) write speed all = 0.658195982 MB/sec (without seq num) 2024/11/23-13:11:53.689905 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260762: 924995 keys, 35297302 bytes, temperature: kWarm 2024/11/23-13:11:53.689995 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338713689940, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260762, "file_size": 35297302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375502381, "largest_seqno": 3990166904, "table_properties": {"data_size": 28971808, "index_size": 1771456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899860, "raw_average_key_size": 28, "raw_value_size": 108318006, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924995, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338709, "oldest_key_time": 0, "file_creation_time": 1732338709, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260762, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:11:53.739383 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst) /dev/shm/ToplingTemp/Topling-56986-g1ZFpd 2024/11/23-13:11:54.594911 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst): first pass time = 0.86's, 156.890'MB/sec 2024/11/23-13:11:57.141774 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst): old prealloc_size = 147639500, real_size = 32432096 2024/11/23-13:11:57.142350 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929649 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.37 avg-zval = 32.73 2024/11/23-13:11:57.143159 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:11:58.224468 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260763.sst): second pass time = 2.55's, 42.496'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.856'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.661'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.275'MB/sec write SST data time = 0.24's, 153.989'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 38029.323'MB/sec zip my value time = 2.55's, unzip length = 0.108'GB zip my value throughput = 42.496'MB/sec zip pipeline throughput = 111.767'MB/sec entries = 929649 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 32.73 usrkeys = 929649 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 32.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929649 bytes = 3370000 } seq expand size = 6507543 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5126 value = 3.5552 dict = 1.00 all = 3.6617 } Zip/UnZip{ index = 0.0951 value = 0.2813 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 9.192763 GB avg = 0.167 KB (by entry num) total key len = 1.518835 GB avg = 0.028 KB total ukey len = 1.053602 GB avg = 0.015 KB total ukey num = 0.068336655 Billion total entry num = 0.055144384 Billion write speed all = 0.695286967 MB/sec (with seq num) write speed all = 0.666651708 MB/sec (without seq num) 2024/11/23-13:11:58.284729 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260763: 929649 keys, 36654548 bytes, temperature: kWarm 2024/11/23-13:11:58.284797 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338718284762, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260763, "file_size": 36654548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3990166922, "table_properties": {"data_size": 30430864, "index_size": 1768640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030172, "raw_average_key_size": 28, "raw_value_size": 108187594, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929649, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338713, "oldest_key_time": 0, "file_creation_time": 1732338713, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260763, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:11:58.322672 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst) /dev/shm/ToplingTemp/Topling-56986-MizlNe 2024/11/23-13:11:59.245634 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst): first pass time = 0.92's, 145.426'MB/sec 2024/11/23-13:12:00.826202 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst): old prealloc_size = 147639500, real_size = 32517840 2024/11/23-13:12:00.826719 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930825 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.19 avg-zval = 32.77 2024/11/23-13:12:00.827538 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:01.859619 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260764.sst): second pass time = 1.58's, 68.463'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 35.556'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3600.777'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.257'MB/sec write SST data time = 0.13's, 289.610'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34665.316'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.463'MB/sec zip pipeline throughput = 135.017'MB/sec entries = 930825 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 32.77 usrkeys = 930825 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.19 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930825 bytes = 3374256 } seq expand size = 6515775 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4443 value = 3.5458 dict = 1.00 all = 3.6528 } Zip/UnZip{ index = 0.0957 value = 0.2820 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 9.307433 GB avg = 0.166 KB (by entry num) total key len = 1.544898 GB avg = 0.028 KB total ukey len = 1.072219 GB avg = 0.015 KB total ukey num = 0.069267480 Billion total entry num = 0.056075209 Billion write speed all = 0.704255772 MB/sec (with seq num) write speed all = 0.675144026 MB/sec (without seq num) 2024/11/23-13:12:01.937905 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260764: 930825 keys, 36743398 bytes, temperature: kWarm 2024/11/23-13:12:01.937992 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338721937940, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260764, "file_size": 36743398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3990169196, "table_properties": {"data_size": 30502464, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063100, "raw_average_key_size": 28, "raw_value_size": 108154718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930825, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338718, "oldest_key_time": 0, "file_creation_time": 1732338718, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260764, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:01.976670 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst) /dev/shm/ToplingTemp/Topling-56986-2dF6Gd 2024/11/23-13:12:03.060116 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst): first pass time = 1.08's, 123.885'MB/sec 2024/11/23-13:12:04.749515 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst): old prealloc_size = 147639500, real_size = 33883616 2024/11/23-13:12:04.750207 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931539 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 116.08 avg-zval = 34.22 2024/11/23-13:12:04.751111 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:05.782205 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260765.sst): second pass time = 1.69's, 64.040'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 37.018'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2649.993'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.16's, 232.789'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35811.875'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.040'MB/sec zip pipeline throughput = 98.472'MB/sec entries = 931539 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.08 avg-zval = 34.22 usrkeys = 931539 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.08 avg-zval = 34.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931539 bytes = 3493280 } seq expand size = 6520773 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5142 value = 3.3921 dict = 1.00 all = 3.5110 } Zip/UnZip{ index = 0.0951 value = 0.2948 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 9.422089 GB avg = 0.165 KB (by entry num) total key len = 1.570981 GB avg = 0.028 KB total ukey len = 1.090850 GB avg = 0.016 KB total ukey num = 0.070199019 Billion total entry num = 0.057006748 Billion write speed all = 0.713207370 MB/sec (with seq num) write speed all = 0.683619542 MB/sec (without seq num) 2024/11/23-13:12:05.831024 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260765: 931539 keys, 38227670 bytes, temperature: kWarm 2024/11/23-13:12:05.831098 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338725831064, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260765, "file_size": 38227670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3990169540, "table_properties": {"data_size": 31878608, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26083092, "raw_average_key_size": 28, "raw_value_size": 108134706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931539, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338721, "oldest_key_time": 0, "file_creation_time": 1732338721, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260765, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:05.866119 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst) /dev/shm/ToplingTemp/Topling-56986-GP8zgb 2024/11/23-13:12:07.284176 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst): first pass time = 1.42's, 94.652'MB/sec 2024/11/23-13:12:08.745316 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst): old prealloc_size = 147639500, real_size = 30271840 2024/11/23-13:12:08.745868 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931216 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 116.13 avg-zval = 30.34 2024/11/23-13:12:08.746674 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:09.571814 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260766.sst): second pass time = 1.46's, 74.054'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.740'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.952'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 273.049'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34237.300'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.054'MB/sec zip pipeline throughput = 104.666'MB/sec entries = 931216 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.13 avg-zval = 30.34 usrkeys = 931216 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.13 avg-zval = 30.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931216 bytes = 3375680 } seq expand size = 6518512 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4449 value = 3.8273 dict = 1.00 all = 3.8907 } Zip/UnZip{ index = 0.0957 value = 0.2613 dict = 1.00 all = 0.2570 } ---------------------------- total value len = 9.536751 GB avg = 0.165 KB (by entry num) total key len = 1.597055 GB avg = 0.028 KB total ukey len = 1.109474 GB avg = 0.016 KB total ukey num = 0.071130235 Billion total entry num = 0.057937964 Billion write speed all = 0.722160499 MB/sec (with seq num) write speed all = 0.692096739 MB/sec (without seq num) 2024/11/23-13:12:09.614796 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260766: 931216 keys, 34497042 bytes, temperature: kWarm 2024/11/23-13:12:09.614857 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338729614825, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260766, "file_size": 34497042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3703182610, "largest_seqno": 3990166961, "table_properties": {"data_size": 28255760, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074048, "raw_average_key_size": 28, "raw_value_size": 108143796, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931216, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338725, "oldest_key_time": 0, "file_creation_time": 1732338725, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260766, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:09.649738 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst) /dev/shm/ToplingTemp/Topling-56986-sVvuPc 2024/11/23-13:12:10.621970 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst): first pass time = 0.97's, 138.057'MB/sec 2024/11/23-13:12:11.980833 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst): old prealloc_size = 147639500, real_size = 34940384 2024/11/23-13:12:11.981415 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927026 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 116.78 avg-zval = 35.53 2024/11/23-13:12:11.982143 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:12.798602 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260767.sst): second pass time = 1.36's, 79.766'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.366'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.247'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 304.016'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36616.610'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.766'MB/sec zip pipeline throughput = 79.766'MB/sec entries = 927026 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.78 avg-zval = 35.53 usrkeys = 927026 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.40 avg-zval = 35.53 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 744029 bytes = 2790128 } seq expand size = 5208203 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4455 value = 3.2873 dict = 1.00 all = 3.4652 } Zip/UnZip{ index = 0.0957 value = 0.3042 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 9.650220 GB avg = 0.164 KB (by entry num) total key len = 1.623012 GB avg = 0.028 KB total ukey len = 1.128015 GB avg = 0.016 KB total ukey num = 0.072057261 Billion total entry num = 0.058864990 Billion write speed all = 0.731050938 MB/sec (with seq num) write speed all = 0.700512541 MB/sec (without seq num) 2024/11/23-13:12:12.846622 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260767: 927026 keys, 38733390 bytes, temperature: kWarm 2024/11/23-13:12:12.846691 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338732846657, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260767, "file_size": 38733390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3990169255, "table_properties": {"data_size": 32933456, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956728, "raw_average_key_size": 28, "raw_value_size": 108261096, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927026, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338729, "oldest_key_time": 0, "file_creation_time": 1732338729, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260767, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:12.885042 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst) /dev/shm/ToplingTemp/Topling-56986-kGizne 2024/11/23-13:12:13.899145 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst): first pass time = 1.01's, 132.358'MB/sec 2024/11/23-13:12:15.259723 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst): old prealloc_size = 147639500, real_size = 33207904 2024/11/23-13:12:15.260329 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935801 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0312 GB avg-val = 115.43 avg-zval = 33.31 2024/11/23-13:12:15.261127 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:16.110139 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260768.sst): second pass time = 1.36's, 79.438'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.038'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3090.950'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 297.113'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33202.453'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.438'MB/sec zip pipeline throughput = 79.438'MB/sec entries = 935801 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.43 avg-zval = 33.31 usrkeys = 935801 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.43 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935801 bytes = 3392288 } seq expand size = 6550607 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3775 value = 3.4653 dict = 1.00 all = 3.5840 } Zip/UnZip{ index = 0.0964 value = 0.2886 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 9.764786 GB avg = 0.163 KB (by entry num) total key len = 1.649215 GB avg = 0.028 KB total ukey len = 1.146731 GB avg = 0.016 KB total ukey num = 0.072993062 Billion total entry num = 0.059800791 Billion write speed all = 0.740020628 MB/sec (with seq num) write speed all = 0.709003410 MB/sec (without seq num) 2024/11/23-13:12:16.158443 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260768: 935801 keys, 37449432 bytes, temperature: kWarm 2024/11/23-13:12:16.158515 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338736158477, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260768, "file_size": 37449432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3990168955, "table_properties": {"data_size": 31170256, "index_size": 1803520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26202428, "raw_average_key_size": 28, "raw_value_size": 108015418, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935801, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338732, "oldest_key_time": 0, "file_creation_time": 1732338732, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260768, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:16.194283 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst) /dev/shm/ToplingTemp/Topling-56986-fnZbnd 2024/11/23-13:12:17.063073 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst): first pass time = 0.87's, 154.495'MB/sec 2024/11/23-13:12:18.358628 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst): old prealloc_size = 147639500, real_size = 30983424 2024/11/23-13:12:18.359199 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928078 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0290 GB avg-val = 116.62 avg-zval = 31.23 2024/11/23-13:12:18.359970 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:19.175326 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260769.sst): second pass time = 1.29's, 83.589'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.104'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.848'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 276.868'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34804.487'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.589'MB/sec zip pipeline throughput = 83.589'MB/sec entries = 928078 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 31.23 usrkeys = 928078 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 31.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928078 bytes = 3364304 } seq expand size = 6496546 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5131 value = 3.7340 dict = 1.00 all = 3.8129 } Zip/UnZip{ index = 0.0951 value = 0.2678 dict = 1.00 all = 0.2623 } ---------------------------- total value len = 9.879514 GB avg = 0.163 KB (by entry num) total key len = 1.675201 GB avg = 0.028 KB total ukey len = 1.165292 GB avg = 0.016 KB total ukey num = 0.073921140 Billion total entry num = 0.060728869 Billion write speed all = 0.748994923 MB/sec (with seq num) write speed all = 0.717502592 MB/sec (without seq num) 2024/11/23-13:12:19.218958 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260769: 928078 keys, 35201396 bytes, temperature: kWarm 2024/11/23-13:12:19.219045 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338739219007, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260769, "file_size": 35201396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3990168973, "table_properties": {"data_size": 28985648, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986184, "raw_average_key_size": 28, "raw_value_size": 108231668, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928078, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338736, "oldest_key_time": 0, "file_creation_time": 1732338736, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260769, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:19.254919 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst) /dev/shm/ToplingTemp/Topling-56986-48yBWc 2024/11/23-13:12:20.267225 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst): first pass time = 1.01's, 132.591'MB/sec 2024/11/23-13:12:21.560997 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst): old prealloc_size = 147639500, real_size = 31641232 2024/11/23-13:12:21.561537 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931321 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.12 avg-zval = 31.81 2024/11/23-13:12:21.563634 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:22.396812 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260770.sst): second pass time = 1.29's, 83.636'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.054'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.958'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 283.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33971.201'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.636'MB/sec zip pipeline throughput = 83.636'MB/sec entries = 931321 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.12 avg-zval = 31.81 usrkeys = 931321 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.12 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931321 bytes = 3492464 } seq expand size = 6519247 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4438 value = 3.6504 dict = 1.00 all = 3.7298 } Zip/UnZip{ index = 0.0958 value = 0.2739 dict = 1.00 all = 0.2681 } ---------------------------- total value len = 9.994175 GB avg = 0.162 KB (by entry num) total key len = 1.701278 GB avg = 0.028 KB total ukey len = 1.183919 GB avg = 0.016 KB total ukey num = 0.074852461 Billion total entry num = 0.061660190 Billion write speed all = 0.757959447 MB/sec (with seq num) write speed all = 0.725990834 MB/sec (without seq num) 2024/11/23-13:12:22.439929 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260770: 931321 keys, 35984848 bytes, temperature: kWarm 2024/11/23-13:12:22.440015 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338742439960, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260770, "file_size": 35984848, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3990168986, "table_properties": {"data_size": 29624704, "index_size": 1783488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076988, "raw_average_key_size": 28, "raw_value_size": 108140878, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931321, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338739, "oldest_key_time": 0, "file_creation_time": 1732338739, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260770, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:22.478140 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst) /dev/shm/ToplingTemp/Topling-56986-ex73Sb 2024/11/23-13:12:23.501859 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst): first pass time = 1.02's, 131.112'MB/sec 2024/11/23-13:12:24.788467 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst): old prealloc_size = 147639500, real_size = 31542496 2024/11/23-13:12:24.789022 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925004 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 117.10 avg-zval = 31.95 2024/11/23-13:12:24.790773 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:25.609950 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260771.sst): second pass time = 1.29's, 84.263'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.522'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.843'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 278.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34454.580'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.263'MB/sec zip pipeline throughput = 84.263'MB/sec entries = 925004 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 31.95 usrkeys = 925004 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925004 bytes = 3353152 } seq expand size = 6475028 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5114 value = 3.6654 dict = 1.00 all = 3.7542 } Zip/UnZip{ index = 0.0951 value = 0.2728 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 10.108967 GB avg = 0.162 KB (by entry num) total key len = 1.727178 GB avg = 0.028 KB total ukey len = 1.202419 GB avg = 0.016 KB total ukey num = 0.075777465 Billion total entry num = 0.062585194 Billion write speed all = 0.766917776 MB/sec (with seq num) write speed all = 0.734476337 MB/sec (without seq num) 2024/11/23-13:12:25.654486 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260771: 925004 keys, 35751202 bytes, temperature: kWarm 2024/11/23-13:12:25.654555 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338745654520, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260771, "file_size": 35751202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3990169006, "table_properties": {"data_size": 29551056, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900112, "raw_average_key_size": 28, "raw_value_size": 108317728, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925004, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338742, "oldest_key_time": 0, "file_creation_time": 1732338742, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260771, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:25.690321 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst) /dev/shm/ToplingTemp/Topling-56986-DdifMc 2024/11/23-13:12:26.678905 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst): first pass time = 0.99's, 135.772'MB/sec 2024/11/23-13:12:28.000218 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst): old prealloc_size = 147639500, real_size = 33093792 2024/11/23-13:12:28.000720 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926671 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.84 avg-zval = 33.56 2024/11/23-13:12:28.002454 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:28.850838 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260772.sst): second pass time = 1.32's, 81.990'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3689.301'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 293.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38544.319'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.990'MB/sec zip pipeline throughput = 81.990'MB/sec entries = 926671 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 33.56 usrkeys = 926671 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.84 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926671 bytes = 3359200 } seq expand size = 6486697 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5151 value = 3.4814 dict = 1.00 all = 3.5976 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 10.223725 GB avg = 0.161 KB (by entry num) total key len = 1.753125 GB avg = 0.028 KB total ukey len = 1.220952 GB avg = 0.016 KB total ukey num = 0.076704136 Billion total entry num = 0.063511865 Billion write speed all = 0.775871732 MB/sec (with seq num) write speed all = 0.742956859 MB/sec (without seq num) 2024/11/23-13:12:28.895213 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260772: 926671 keys, 37307318 bytes, temperature: kWarm 2024/11/23-13:12:28.895282 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338748895247, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260772, "file_size": 37307318, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3596991077, "largest_seqno": 3990166090, "table_properties": {"data_size": 31099408, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946788, "raw_average_key_size": 28, "raw_value_size": 108270962, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926671, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338745, "oldest_key_time": 0, "file_creation_time": 1732338745, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260772, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:28.931687 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst) /dev/shm/ToplingTemp/Topling-56986-P9zqKe 2024/11/23-13:12:29.963712 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst): first pass time = 1.03's, 130.058'MB/sec 2024/11/23-13:12:31.265127 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst): old prealloc_size = 147639500, real_size = 32579648 2024/11/23-13:12:31.265647 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933528 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.77 avg-zval = 32.75 2024/11/23-13:12:31.267396 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:32.133221 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260773.sst): second pass time = 1.30's, 83.098'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.544'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3589.571'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 285.099'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33689.625'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.098'MB/sec zip pipeline throughput = 83.098'MB/sec entries = 933528 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.77 avg-zval = 32.75 usrkeys = 933528 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.77 avg-zval = 32.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933528 bytes = 3500752 } seq expand size = 6534696 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5112 value = 3.5355 dict = 1.00 all = 3.6343 } Zip/UnZip{ index = 0.0951 value = 0.2828 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 10.338339 GB avg = 0.160 KB (by entry num) total key len = 1.779263 GB avg = 0.028 KB total ukey len = 1.239623 GB avg = 0.016 KB total ukey num = 0.077637664 Billion total entry num = 0.064445393 Billion write speed all = 0.784822930 MB/sec (with seq num) write speed all = 0.751431358 MB/sec (without seq num) 2024/11/23-13:12:32.178935 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260773: 933528 keys, 36931012 bytes, temperature: kWarm 2024/11/23-13:12:32.179020 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338752178968, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260773, "file_size": 36931012, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3990169030, "table_properties": {"data_size": 30569840, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26138784, "raw_average_key_size": 28, "raw_value_size": 108079000, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933528, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338748, "oldest_key_time": 0, "file_creation_time": 1732338748, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260773, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:32.220842 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst) /dev/shm/ToplingTemp/Topling-56986-YTAE1e 2024/11/23-13:12:32.844127 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst): first pass time = 0.62's, 215.352'MB/sec 2024/11/23-13:12:34.149417 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst): old prealloc_size = 147639500, real_size = 33009696 2024/11/23-13:12:34.149965 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926746 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.83 avg-zval = 33.44 2024/11/23-13:12:34.151742 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-13:12:35.003728 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260774.sst): second pass time = 1.30's, 82.995'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.625'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 286.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27226.267'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.995'MB/sec zip pipeline throughput = 82.995'MB/sec entries = 926746 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.83 avg-zval = 33.44 usrkeys = 926746 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.83 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926746 bytes = 3359472 } seq expand size = 6487222 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.3758 value = 3.4935 dict = 1.00 all = 3.6057 } Zip/UnZip{ index = 0.0964 value = 0.2862 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 10.453095 GB avg = 0.160 KB (by entry num) total key len = 1.805212 GB avg = 0.028 KB total ukey len = 1.258158 GB avg = 0.016 KB total ukey num = 0.078564410 Billion total entry num = 0.065372139 Billion write speed all = 0.793788422 MB/sec (with seq num) write speed all = 0.759922965 MB/sec (without seq num) 2024/11/23-13:12:35.052827 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260774: 926746 keys, 37223378 bytes, temperature: kWarm 2024/11/23-13:12:35.052898 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338755052863, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260774, "file_size": 37223378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3990168018, "table_properties": {"data_size": 30991440, "index_size": 1786368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948888, "raw_average_key_size": 28, "raw_value_size": 108268936, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926746, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338752, "oldest_key_time": 0, "file_creation_time": 1732338752, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260774, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:35.091309 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst) /dev/shm/ToplingTemp/Topling-56986-WebMcb 2024/11/23-13:12:35.147139 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst): first pass time = 0.06's, 235.981'MB/sec 2024/11/23-13:12:35.264413 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst): old prealloc_size = 147639500, real_size = 3799408 2024/11/23-13:12:35.264518 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 92345 prefix = 4 raw-key = 0.0015 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0106 GB zip-val = 0.0036 GB avg-val = 114.57 avg-zval = 38.96 2024/11/23-13:12:35.264627 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.013 GB 2024/11/23-13:12:35.344971 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00/260775.sst): second pass time = 0.12's, 90.754'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.08's, 51.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2166.259'MB/sec (index lex order gen) rebuild zvType time = 0.07's, 0.342'MB/sec write SST data time = 0.01's, 323.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.109'MB, throughput = 4478.549'MB/sec zip my value time = 0.12's, unzip length = 0.011'GB zip my value throughput = 90.754'MB/sec zip pipeline throughput = 90.754'MB/sec entries = 92345 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.57 avg-zval = 38.96 usrkeys = 92345 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.57 avg-zval = 38.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 92345 bytes = 277056 } seq expand size = 646415 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0018 GB value = 0.0106 GB dict = 0.11 MB all = 0.0132 GB } __ZipSize{ index = 0.0002 GB value = 0.0036 GB dict = 0.11 MB all = 0.0042 GB } UnZip/Zip{ index = 10.3693 value = 2.9406 dict = 1.00 all = 3.1616 } Zip/UnZip{ index = 0.0964 value = 0.3401 dict = 1.00 all = 0.3163 } ---------------------------- total value len = 10.464322 GB avg = 0.160 KB (by entry num) total key len = 1.807798 GB avg = 0.028 KB total ukey len = 1.260004 GB avg = 0.016 KB total ukey num = 0.078656755 Billion total entry num = 0.065464484 Billion write speed all = 0.794665287 MB/sec (with seq num) write speed all = 0.760752741 MB/sec (without seq num) 2024/11/23-13:12:35.356969 139957483730496 [compaction_job.cc:1948] [default] [JOB 26600] Generated table #260775: 92345 keys, 4164329 bytes, temperature: kWarm 2024/11/23-13:12:35.357046 139957483730496 EVENT_LOG_v1 {"time_micros": 1732338755357014, "cf_name": "default", "job": 26600, "event": "table_file_creation", "file_number": 260775, "file_size": 4164329, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3976683785, "largest_seqno": 3990169041, "table_properties": {"data_size": 3598048, "index_size": 178112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2585660, "raw_average_key_size": 28, "raw_value_size": 10580342, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 92345, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732338755, "oldest_key_time": 0, "file_creation_time": 1732338755, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 260775, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-13:12:35.367863 139957483730496 [compaction_job.cc:703] job-26600: subcompact[0], size: 0.516333 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-26600/att-00 [260761,260762,260763,260764,260765,260766,260767,260768,260769,260770,260771,260772,260773,260774,260775] 2024/11/23-13:12:35.455577 139957483730496 (Original Log Time 2024/11/23-13:11:44.842135) EVENT_LOG_v1 {"time_micros": 1732338704842086, "job": 26600, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [260715], "files_L3": [260676, 260677, 260678, 260679, 260680, 260681, 260638, 260639, 260640, 260641, 260642, 260643, 260644, 260645, 260721], "score": 0.998435, "input_data_size": 540952305, "oldest_snapshot_seqno": 3990317844} 2024/11/23-13:12:35.455580 139957483730496 (Original Log Time 2024/11/23-13:12:35.454296) [compaction_job.cc:2022] [default] [JOB 26600] Compacted 1@2 + 15@3 files to L3 => 516332679 bytes 2024/11/23-13:12:35.455581 139957483730496 (Original Log Time 2024/11/23-13:12:35.455513) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 488.8 +0.0 blob) out(492.4 +0.0 blob), read-write-amplify(37.2) write-amplify(18.2) OK, records in: 13152505, records dropped: 46185 output_compression: Snappy 2024/11/23-13:12:35.455587 139957483730496 (Original Log Time 2024/11/23-13:12:35.455556) EVENT_LOG_v1 {"time_micros": 1732338755455536, "cf": "default", "job": 26600, "event": "compaction_finished", "compaction_time_micros": 50525733, "compaction_time_cpu_micros": 20551727, "output_level": 3, "num_output_files": 15, "total_output_size": 516332679, "num_input_records": 13152505, "num_output_records": 13106320, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 381599350, "file_range_sync_nanos": 0, "file_fsync_nanos": 10215, "file_prepare_write_nanos": 46234, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-13:12:35.594109 139957483730496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732338704,"executesMs":50774,"compactionJobId":26600,"attempt":0,"compactionInputRawBytes":1898773726,"compactionInputZipBytes":540952305,"compactionOutputRawBytes":0,"compactionOutputZipBytes":516332679,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-969d12e69bf67e430a7c9ea82741d9ba-cd82f9cf2b7c47a3-00"}}, response = 2024/11/23-13:12:35.594202 139957483730496 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-26600/att-00: olev 3, work 50.775 s, result 6.028 ms, install 1.310 ms, input{raw 1.768 GiB zip 515.892 MiB}