2024/11/23-15:39:13.852367 139957592835648 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00: fork to child time = 0.007647 sec 2024/11/23-15:39:13.853173 139957592835648 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00 2024/11/23-15:39:13.860416 139957592835648 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00 2024/11/23-15:39:13.861359 139957592835648 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:39:13.861632 139957592835648 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:39:13.921192 139957592835648 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29569/att-00/MANIFEST-206347 2024/11/23-15:39:13.955359 139957592835648 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29569/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 267116, last_sequence is 4076186805, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 266953 2024/11/23-15:39:13.955371 139957592835648 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:39:13.955506 139957592835648 [version_set.cc:5721] Creating manifest 267116 2024/11/23-15:39:13.992355 139957592835648 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:39:14.014353 139957592835648 [compaction_job.cc:2437] [default] [JOB 29569] Compacting 1@2 + 24@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-15:39:14.014372 139957592835648 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 22101 Base level 2, inputs: [267082(27MB)], [267106(39MB) 267107(4184KB) 267029(35MB) 267030(34MB) 267031(35MB) 267032(34MB) 267033(36MB) 267034(34MB) 267035(34MB) 267036(34MB) 267037(35MB) 267038(38MB) 267039(36MB) 267040(35MB) 267041(34MB) 267042(36MB) 267043(34MB) 267044(37MB) 267045(33MB) 267046(34MB) 267047(36MB) 267048(37MB) 267056(38MB) 267108(37MB)] 2024/11/23-15:39:14.021798 139957592835648 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:39:14.021828 139957592835648 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:39:14.021847 139957592835648 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:39:14.021853 139957592835648 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:39:14.021862 139957592835648 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:39:14.649312 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst) /dev/shm/ToplingTemp/Topling-98436-L1LBFu 2024/11/23-15:39:15.422309 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst): first pass time = 0.77's, 173.648'MB/sec 2024/11/23-15:39:17.128956 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst): old prealloc_size = 147639500, real_size = 37428384 2024/11/23-15:39:17.129485 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930704 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0354 GB avg-val = 116.21 avg-zval = 38.06 2024/11/23-15:39:17.130319 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:17.949610 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267117.sst): second pass time = 1.71's, 63.408'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 50.763'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3548.586'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 330.369'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38983.552'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.408'MB/sec zip pipeline throughput = 63.408'MB/sec entries = 930704 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.21 avg-zval = 38.06 usrkeys = 930704 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.21 avg-zval = 38.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930704 bytes = 3373824 } seq expand size = 6514928 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0354 GB dict = 1.08 MB all = 0.0417 GB } UnZip/Zip{ index = 10.5131 value = 3.0532 dict = 1.00 all = 3.2219 } Zip/UnZip{ index = 0.0951 value = 0.3275 dict = 1.00 all = 0.3104 } ---------------------------- total value len = 8.963293 GB avg = 0.168 KB (by entry num) total key len = 1.466881 GB avg = 0.028 KB total ukey len = 1.016492 GB avg = 0.015 KB total ukey num = 0.066481139 Billion total entry num = 0.053288868 Billion write speed all = 0.430185909 MB/sec (with seq num) write speed all = 0.412602984 MB/sec (without seq num) 2024/11/23-15:39:17.999622 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267117: 930704 keys, 41657554 bytes, temperature: kWarm 2024/11/23-15:39:17.999750 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347557999665, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267117, "file_size": 41657554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4075878629, "table_properties": {"data_size": 35424976, "index_size": 1770560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059712, "raw_average_key_size": 28, "raw_value_size": 108158028, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347554, "oldest_key_time": 0, "file_creation_time": 1732347554, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267117, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:18.045954 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst) /dev/shm/ToplingTemp/Topling-98436-g2Zk6t 2024/11/23-15:39:19.088994 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst): first pass time = 1.04's, 128.689'MB/sec 2024/11/23-15:39:20.563683 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst): old prealloc_size = 147639500, real_size = 33210336 2024/11/23-15:39:20.564407 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929127 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.46 avg-zval = 33.58 2024/11/23-15:39:20.565320 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:21.458143 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267118.sst): second pass time = 1.47's, 73.418'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2530.754'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.15's, 257.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36451.184'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.418'MB/sec zip pipeline throughput = 73.418'MB/sec entries = 929127 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.46 avg-zval = 33.58 usrkeys = 929127 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.46 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929127 bytes = 3484240 } seq expand size = 6503889 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4451 value = 3.4682 dict = 1.00 all = 3.5746 } Zip/UnZip{ index = 0.0957 value = 0.2883 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 9.077999 GB avg = 0.167 KB (by entry num) total key len = 1.492896 GB avg = 0.028 KB total ukey len = 1.035075 GB avg = 0.015 KB total ukey num = 0.067410266 Billion total entry num = 0.054217995 Billion write speed all = 0.435926803 MB/sec (with seq num) write speed all = 0.418039896 MB/sec (without seq num) 2024/11/23-15:39:21.503721 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267118: 929127 keys, 37547468 bytes, temperature: kWarm 2024/11/23-15:39:21.503787 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347561503756, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267118, "file_size": 37547468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776149, "table_properties": {"data_size": 31198800, "index_size": 1779072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26015556, "raw_average_key_size": 28, "raw_value_size": 108202190, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929127, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347558, "oldest_key_time": 0, "file_creation_time": 1732347558, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267118, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:21.540094 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst) /dev/shm/ToplingTemp/Topling-98436-LErOQs 2024/11/23-15:39:22.767437 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst): first pass time = 1.22's, 110.112'MB/sec 2024/11/23-15:39:24.296410 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst): old prealloc_size = 147639500, real_size = 32636624 2024/11/23-15:39:24.297004 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931619 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.07 avg-zval = 32.87 2024/11/23-15:39:24.297769 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:25.156374 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267119.sst): second pass time = 1.53's, 70.753'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.734'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3111.999'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.12's, 297.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31393.122'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.753'MB/sec zip pipeline throughput = 70.753'MB/sec entries = 931619 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 32.87 usrkeys = 931619 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931619 bytes = 3260688 } seq expand size = 6521333 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4442 value = 3.5315 dict = 1.00 all = 3.6523 } Zip/UnZip{ index = 0.0957 value = 0.2832 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 9.192653 GB avg = 0.167 KB (by entry num) total key len = 1.518982 GB avg = 0.028 KB total ukey len = 1.053707 GB avg = 0.015 KB total ukey num = 0.068341885 Billion total entry num = 0.055149614 Billion write speed all = 0.441663300 MB/sec (with seq num) write speed all = 0.423471820 MB/sec (without seq num) 2024/11/23-15:39:25.204288 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267119: 931619 keys, 36748358 bytes, temperature: kWarm 2024/11/23-15:39:25.204354 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347565204321, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267119, "file_size": 36748358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776165, "table_properties": {"data_size": 30619520, "index_size": 1784000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085332, "raw_average_key_size": 28, "raw_value_size": 108132434, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347561, "oldest_key_time": 0, "file_creation_time": 1732347561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267119, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:25.239136 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst) /dev/shm/ToplingTemp/Topling-98436-kX3gQt 2024/11/23-15:39:26.363055 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst): first pass time = 1.12's, 119.428'MB/sec 2024/11/23-15:39:27.844654 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst): old prealloc_size = 147639500, real_size = 32965072 2024/11/23-15:39:27.845380 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930832 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.19 avg-zval = 33.25 2024/11/23-15:39:27.846255 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:28.740163 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267120.sst): second pass time = 1.48's, 73.044'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.401'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2527.307'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.307'MB/sec write SST data time = 0.14's, 272.755'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37127.578'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.044'MB/sec zip pipeline throughput = 73.044'MB/sec entries = 930832 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 33.25 usrkeys = 930832 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.19 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930832 bytes = 3257920 } seq expand size = 6515824 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4466 value = 3.4945 dict = 1.00 all = 3.6202 } Zip/UnZip{ index = 0.0957 value = 0.2862 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 9.307323 GB avg = 0.166 KB (by entry num) total key len = 1.545045 GB avg = 0.028 KB total ukey len = 1.072324 GB avg = 0.015 KB total ukey num = 0.069272717 Billion total entry num = 0.056080446 Billion write speed all = 0.447399933 MB/sec (with seq num) write speed all = 0.428904144 MB/sec (without seq num) 2024/11/23-15:39:28.787842 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267120: 930832 keys, 37074398 bytes, temperature: kWarm 2024/11/23-15:39:28.787912 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347568787876, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267120, "file_size": 37074398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30950080, "index_size": 1782080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063296, "raw_average_key_size": 28, "raw_value_size": 108154492, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347565, "oldest_key_time": 0, "file_creation_time": 1732347565, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:28.824634 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst) /dev/shm/ToplingTemp/Topling-98436-zAuV2u 2024/11/23-15:39:29.830610 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst): first pass time = 1.01's, 133.426'MB/sec 2024/11/23-15:39:31.647910 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst): old prealloc_size = 147639500, real_size = 34225600 2024/11/23-15:39:31.648480 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935373 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0322 GB avg-val = 115.49 avg-zval = 34.43 2024/11/23-15:39:31.648952 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:32.423713 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267121.sst): second pass time = 1.82's, 59.468'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 47.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.317'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.361'MB/sec write SST data time = 0.13's, 289.478'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34671.986'MB/sec zip my value time = 1.82's, unzip length = 0.108'GB zip my value throughput = 59.468'MB/sec zip pipeline throughput = 115.586'MB/sec entries = 935373 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.49 avg-zval = 34.43 usrkeys = 935373 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.77 avg-zval = 34.43 TagRS{ kinds = RS_Key0_TagN bytes = 146240 } TagArray{ size = 438185 bytes = 1478896 } seq expand size = 3067295 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4439 value = 3.3548 dict = 1.00 all = 3.6568 } Zip/UnZip{ index = 0.0957 value = 0.2981 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 9.418418 GB avg = 0.165 KB (by entry num) total key len = 1.571235 GB avg = 0.028 KB total ukey len = 1.091031 GB avg = 0.016 KB total ukey num = 0.070208090 Billion total entry num = 0.057015819 Billion write speed all = 0.452990859 MB/sec (with seq num) write speed all = 0.434189432 MB/sec (without seq num) 2024/11/23-15:39:32.470760 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267121: 935373 keys, 36704087 bytes, temperature: kWarm 2024/11/23-15:39:32.470830 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347572470795, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267121, "file_size": 36704087, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32200368, "index_size": 1791232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26190444, "raw_average_key_size": 28, "raw_value_size": 108027318, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935373, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347568, "oldest_key_time": 0, "file_creation_time": 1732347568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267121, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:32.508368 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst) /dev/shm/ToplingTemp/Topling-98436-9gYdbv 2024/11/23-15:39:33.387179 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst): first pass time = 0.88's, 152.733'MB/sec 2024/11/23-15:39:34.721487 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst): old prealloc_size = 147639500, real_size = 35420704 2024/11/23-15:39:34.722137 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080053 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.27 avg-zval = 30.63 2024/11/23-15:39:34.722430 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:35.426421 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267122.sst): second pass time = 1.33's, 77.973'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 52.514'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.690'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.479'MB/sec write SST data time = 0.14's, 262.808'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.026'MB, throughput = 26817.491'MB/sec zip my value time = 1.33's, unzip length = 0.104'GB zip my value throughput = 77.973'MB/sec zip pipeline throughput = 77.973'MB/sec entries = 1080053 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.27 avg-zval = 30.63 usrkeys = 1080053 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.49 avg-zval = 30.63 TagRS{ kinds = RS_Key0_TagN bytes = 168840 } TagArray{ size = 188838 bytes = 660944 } seq expand size = 1321866 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4475 value = 3.1429 dict = 1.00 all = 3.6258 } Zip/UnZip{ index = 0.0957 value = 0.3182 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 9.523716 GB avg = 0.164 KB (by entry num) total key len = 1.601477 GB avg = 0.028 KB total ukey len = 1.112632 GB avg = 0.016 KB total ukey num = 0.071288143 Billion total entry num = 0.058095872 Billion write speed all = 0.458521020 MB/sec (with seq num) write speed all = 0.439365808 MB/sec (without seq num) 2024/11/23-15:39:35.472581 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267122: 1080053 keys, 37017734 bytes, temperature: kWarm 2024/11/23-15:39:35.472656 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347575472621, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267122, "file_size": 37017734, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3417723030, "table_properties": {"data_size": 33082896, "index_size": 2067584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30241484, "raw_average_key_size": 28, "raw_value_size": 103976327, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080053, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347572, "oldest_key_time": 0, "file_creation_time": 1732347572, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267122, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:35.520125 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst) /dev/shm/ToplingTemp/Topling-98436-5A3Gxr 2024/11/23-15:39:36.645853 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst): first pass time = 1.13's, 119.231'MB/sec 2024/11/23-15:39:38.208589 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst): old prealloc_size = 147639500, real_size = 30562688 2024/11/23-15:39:38.209214 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0286 GB avg-val = 116.10 avg-zval = 30.66 2024/11/23-15:39:38.211150 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:39.037302 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267123.sst): second pass time = 1.56's, 69.233'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.123'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3037.837'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 269.348'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34958.918'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.233'MB/sec zip pipeline throughput = 105.367'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 30.66 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 30.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3492880 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5141 value = 3.7866 dict = 1.00 all = 3.8450 } Zip/UnZip{ index = 0.0951 value = 0.2641 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 9.638373 GB avg = 0.163 KB (by entry num) total key len = 1.627557 GB avg = 0.028 KB total ukey len = 1.131261 GB avg = 0.016 KB total ukey num = 0.072219572 Billion total entry num = 0.059027301 Billion write speed all = 0.464252389 MB/sec (with seq num) write speed all = 0.444792965 MB/sec (without seq num) 2024/11/23-15:39:39.081952 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267123: 931429 keys, 34906728 bytes, temperature: kWarm 2024/11/23-15:39:39.082036 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347579082000, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267123, "file_size": 34906728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3417723031, "largest_seqno": 4012775885, "table_properties": {"data_size": 28557872, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347575, "oldest_key_time": 0, "file_creation_time": 1732347575, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:39.119884 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst) /dev/shm/ToplingTemp/Topling-98436-HhD5yt 2024/11/23-15:39:40.049377 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst): first pass time = 0.93's, 144.405'MB/sec 2024/11/23-15:39:41.323216 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst): old prealloc_size = 147639500, real_size = 30234912 2024/11/23-15:39:41.323776 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0282 GB avg-val = 116.12 avg-zval = 30.30 2024/11/23-15:39:41.324564 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:42.147303 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267124.sst): second pass time = 1.27's, 84.948'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.678'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3329.563'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.13's, 267.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32077.346'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.948'MB/sec zip pipeline throughput = 84.948'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.12 avg-zval = 30.30 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.12 avg-zval = 30.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.4441 value = 3.8323 dict = 1.00 all = 3.9079 } Zip/UnZip{ index = 0.0957 value = 0.2609 dict = 1.00 all = 0.2559 } ---------------------------- total value len = 9.753034 GB avg = 0.163 KB (by entry num) total key len = 1.653634 GB avg = 0.028 KB total ukey len = 1.149887 GB avg = 0.016 KB total ukey num = 0.073150881 Billion total entry num = 0.059958610 Billion write speed all = 0.469991719 MB/sec (with seq num) write speed all = 0.450227804 MB/sec (without seq num) 2024/11/23-15:39:42.190394 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267124: 931309 keys, 34345688 bytes, temperature: kWarm 2024/11/23-15:39:42.190468 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347582190432, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267124, "file_size": 34345688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3797357014, "largest_seqno": 4012775902, "table_properties": {"data_size": 28218448, "index_size": 1783424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347579, "oldest_key_time": 0, "file_creation_time": 1732347579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:42.229049 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst) /dev/shm/ToplingTemp/Topling-98436-gDDwor 2024/11/23-15:39:43.156197 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst): first pass time = 0.93's, 144.770'MB/sec 2024/11/23-15:39:44.443767 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst): old prealloc_size = 147639500, real_size = 30754928 2024/11/23-15:39:44.444344 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 116.09 avg-zval = 30.87 2024/11/23-15:39:44.445195 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:45.271222 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267125.sst): second pass time = 1.29's, 84.090'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.419'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.908'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 270.254'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33814.692'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.090'MB/sec zip pipeline throughput = 84.090'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 30.87 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 30.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5125 value = 3.7613 dict = 1.00 all = 3.8240 } Zip/UnZip{ index = 0.0951 value = 0.2659 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 9.867691 GB avg = 0.162 KB (by entry num) total key len = 1.679715 GB avg = 0.028 KB total ukey len = 1.168516 GB avg = 0.016 KB total ukey num = 0.074082343 Billion total entry num = 0.060890072 Billion write speed all = 0.475729349 MB/sec (with seq num) write speed all = 0.455660983 MB/sec (without seq num) 2024/11/23-15:39:45.314879 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267125: 931462 keys, 35099024 bytes, temperature: kWarm 2024/11/23-15:39:45.314949 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347585314914, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267125, "file_size": 35099024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 28749792, "index_size": 1772096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347582, "oldest_key_time": 0, "file_creation_time": 1732347582, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:45.356501 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst) /dev/shm/ToplingTemp/Topling-98436-7TUsHu 2024/11/23-15:39:46.386161 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst): first pass time = 1.03's, 130.357'MB/sec 2024/11/23-15:39:47.669188 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst): old prealloc_size = 147639500, real_size = 31976896 2024/11/23-15:39:47.669747 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924389 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 117.20 avg-zval = 32.44 2024/11/23-15:39:47.670563 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:48.494550 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267126.sst): second pass time = 1.28's, 84.533'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3279.343'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 285.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36277.800'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.533'MB/sec zip pipeline throughput = 84.533'MB/sec entries = 924389 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 32.44 usrkeys = 924389 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 32.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924389 bytes = 3466480 } seq expand size = 6470723 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5140 value = 3.6127 dict = 1.00 all = 3.6975 } Zip/UnZip{ index = 0.0951 value = 0.2768 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 9.982496 GB avg = 0.161 KB (by entry num) total key len = 1.705597 GB avg = 0.028 KB total ukey len = 1.187004 GB avg = 0.016 KB total ukey num = 0.075006732 Billion total entry num = 0.061814461 Billion write speed all = 0.481461490 MB/sec (with seq num) write speed all = 0.461091166 MB/sec (without seq num) 2024/11/23-15:39:48.539657 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267126: 924389 keys, 36299868 bytes, temperature: kWarm 2024/11/23-15:39:48.539735 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347588539694, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267126, "file_size": 36299868, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3279897388, "largest_seqno": 4012775935, "table_properties": {"data_size": 29987184, "index_size": 1758400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882892, "raw_average_key_size": 28, "raw_value_size": 108334878, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347585, "oldest_key_time": 0, "file_creation_time": 1732347585, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:48.576859 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst) /dev/shm/ToplingTemp/Topling-98436-L11lMu 2024/11/23-15:39:49.515669 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst): first pass time = 0.94's, 142.972'MB/sec 2024/11/23-15:39:50.851079 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst): old prealloc_size = 147639500, real_size = 35976640 2024/11/23-15:39:50.851649 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925167 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.07 avg-zval = 36.72 2024/11/23-15:39:50.852425 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:51.680555 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267127.sst): second pass time = 1.33's, 81.156'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 48.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.084'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 319.999'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28556.014'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.156'MB/sec zip pipeline throughput = 81.156'MB/sec entries = 925167 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 36.72 usrkeys = 925167 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 36.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925167 bytes = 3353744 } seq expand size = 6476169 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4434 value = 3.1882 dict = 1.00 all = 3.3399 } Zip/UnZip{ index = 0.0958 value = 0.3137 dict = 1.00 all = 0.2994 } ---------------------------- total value len = 10.097286 GB avg = 0.161 KB (by entry num) total key len = 1.731502 GB avg = 0.028 KB total ukey len = 1.205507 GB avg = 0.016 KB total ukey num = 0.075931899 Billion total entry num = 0.062739628 Billion write speed all = 0.487193082 MB/sec (with seq num) write speed all = 0.466520591 MB/sec (without seq num) 2024/11/23-15:39:51.729408 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267127: 925167 keys, 40185636 bytes, temperature: kWarm 2024/11/23-15:39:51.729480 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347591729444, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267127, "file_size": 40185636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33973360, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904676, "raw_average_key_size": 28, "raw_value_size": 108313086, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925167, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347588, "oldest_key_time": 0, "file_creation_time": 1732347588, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:51.768873 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst) /dev/shm/ToplingTemp/Topling-98436-w2enet 2024/11/23-15:39:52.737110 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst): first pass time = 0.97's, 138.626'MB/sec 2024/11/23-15:39:54.062457 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst): old prealloc_size = 147639500, real_size = 33148112 2024/11/23-15:39:54.063024 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929427 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.41 avg-zval = 33.51 2024/11/23-15:39:54.063877 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:54.912245 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267128.sst): second pass time = 1.32's, 81.706'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.192'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34285.006'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.706'MB/sec zip pipeline throughput = 81.706'MB/sec entries = 929427 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.51 usrkeys = 929427 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 33.51 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 897499 bytes = 3365632 } seq expand size = 6282493 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5143 value = 3.4736 dict = 1.00 all = 3.5777 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 10.211762 GB avg = 0.160 KB (by entry num) total key len = 1.757526 GB avg = 0.028 KB total ukey len = 1.224096 GB avg = 0.016 KB total ukey num = 0.076861326 Billion total entry num = 0.063669055 Billion write speed all = 0.492914267 MB/sec (with seq num) write speed all = 0.471938326 MB/sec (without seq num) 2024/11/23-15:39:54.957837 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267128: 929427 keys, 37515532 bytes, temperature: kWarm 2024/11/23-15:39:54.957906 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347594957871, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267128, "file_size": 37515532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 31147648, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023956, "raw_average_key_size": 28, "raw_value_size": 108193830, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347591, "oldest_key_time": 0, "file_creation_time": 1732347591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267128, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:54.996534 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst) /dev/shm/ToplingTemp/Topling-98436-HTNFUq 2024/11/23-15:39:56.009683 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst): first pass time = 1.01's, 132.480'MB/sec 2024/11/23-15:39:57.346584 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst): old prealloc_size = 147639500, real_size = 32733616 2024/11/23-15:39:57.347145 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930222 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.29 avg-zval = 33.04 2024/11/23-15:39:57.347891 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:39:58.170371 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267129.sst): second pass time = 1.34's, 80.961'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.499'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3352.485'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 287.514'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35977.787'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.961'MB/sec zip pipeline throughput = 80.961'MB/sec entries = 930222 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 33.04 usrkeys = 930222 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.15 avg-zval = 33.04 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 779153 bytes = 2921840 } seq expand size = 5454071 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5141 value = 3.5199 dict = 1.00 all = 3.6615 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 10.325388 GB avg = 0.160 KB (by entry num) total key len = 1.783572 GB avg = 0.028 KB total ukey len = 1.242700 GB avg = 0.016 KB total ukey num = 0.077791548 Billion total entry num = 0.064599277 Billion write speed all = 0.498599275 MB/sec (with seq num) write speed all = 0.477319725 MB/sec (without seq num) 2024/11/23-15:39:58.215456 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267129: 930222 keys, 36656622 bytes, temperature: kWarm 2024/11/23-15:39:58.215525 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347598215490, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267129, "file_size": 36656622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30731424, "index_size": 1769472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046216, "raw_average_key_size": 28, "raw_value_size": 108171536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930222, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347594, "oldest_key_time": 0, "file_creation_time": 1732347594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267129, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:39:58.249923 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst) /dev/shm/ToplingTemp/Topling-98436-tZElQu 2024/11/23-15:39:59.263826 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst): first pass time = 1.01's, 132.382'MB/sec 2024/11/23-15:40:00.588729 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst): old prealloc_size = 147639500, real_size = 31132576 2024/11/23-15:40:00.589282 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.06 avg-zval = 31.26 2024/11/23-15:40:00.590106 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:01.422071 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267130.sst): second pass time = 1.32's, 81.662'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.434'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.002'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 278.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35213.744'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.662'MB/sec zip pipeline throughput = 81.662'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 31.26 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 31.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5136 value = 3.7124 dict = 1.00 all = 3.7957 } Zip/UnZip{ index = 0.0951 value = 0.2694 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 10.440040 GB avg = 0.159 KB (by entry num) total key len = 1.809660 GB avg = 0.028 KB total ukey len = 1.261334 GB avg = 0.016 KB total ukey num = 0.078723240 Billion total entry num = 0.065530969 Billion write speed all = 0.504326853 MB/sec (with seq num) write speed all = 0.482743286 MB/sec (without seq num) 2024/11/23-15:40:01.466017 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267130: 931692 keys, 35360936 bytes, temperature: kWarm 2024/11/23-15:40:01.466085 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347601466050, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267130, "file_size": 35360936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29127120, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347598, "oldest_key_time": 0, "file_creation_time": 1732347598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267130, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:01.503049 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst) /dev/shm/ToplingTemp/Topling-98436-KbLIVt 2024/11/23-15:40:02.370945 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst): first pass time = 0.87's, 154.653'MB/sec 2024/11/23-15:40:03.659502 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst): old prealloc_size = 147639500, real_size = 32726288 2024/11/23-15:40:03.660092 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.82 avg-zval = 32.90 2024/11/23-15:40:03.660916 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:04.516429 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267131.sst): second pass time = 1.29's, 83.934'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.132'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3120.269'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 293.635'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 39020.376'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.934'MB/sec zip pipeline throughput = 83.934'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 32.90 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4448 value = 3.5201 dict = 1.00 all = 3.6314 } Zip/UnZip{ index = 0.0957 value = 0.2841 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 10.554660 GB avg = 0.159 KB (by entry num) total key len = 1.835789 GB avg = 0.028 KB total ukey len = 1.279998 GB avg = 0.016 KB total ukey num = 0.079656450 Billion total entry num = 0.066464179 Billion write speed all = 0.510056637 MB/sec (with seq num) write speed all = 0.488168491 MB/sec (without seq num) 2024/11/23-15:40:04.562149 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267131: 933210 keys, 36959852 bytes, temperature: kWarm 2024/11/23-15:40:04.562221 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347604562185, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267131, "file_size": 36959852, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 30705856, "index_size": 1786944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347601, "oldest_key_time": 0, "file_creation_time": 1732347601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267131, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:04.602024 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst) /dev/shm/ToplingTemp/Topling-98436-0B6HWt 2024/11/23-15:40:05.587353 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst): first pass time = 0.99's, 136.222'MB/sec 2024/11/23-15:40:06.898428 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst): old prealloc_size = 147639500, real_size = 33176816 2024/11/23-15:40:06.898954 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0312 GB avg-val = 117.42 avg-zval = 33.78 2024/11/23-15:40:06.899774 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:07.746139 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267132.sst): second pass time = 1.31's, 82.712'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3520.503'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 293.717'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34824.854'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.712'MB/sec zip pipeline throughput = 82.712'MB/sec entries = 922990 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.42 avg-zval = 33.78 usrkeys = 922990 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.42 avg-zval = 33.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922990 bytes = 3461232 } seq expand size = 6460930 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4430 value = 3.4760 dict = 1.00 all = 3.5795 } Zip/UnZip{ index = 0.0958 value = 0.2877 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 10.669495 GB avg = 0.158 KB (by entry num) total key len = 1.861633 GB avg = 0.028 KB total ukey len = 1.298458 GB avg = 0.016 KB total ukey num = 0.080579440 Billion total entry num = 0.067387169 Billion write speed all = 0.515779144 MB/sec (with seq num) write speed all = 0.493589987 MB/sec (without seq num) 2024/11/23-15:40:07.792236 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267132: 922990 keys, 37495916 bytes, temperature: kWarm 2024/11/23-15:40:07.792304 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347607792269, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267132, "file_size": 37495916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 31178208, "index_size": 1767680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25843720, "raw_average_key_size": 28, "raw_value_size": 108374040, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347604, "oldest_key_time": 0, "file_creation_time": 1732347604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267132, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:07.832887 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst) /dev/shm/ToplingTemp/Topling-98436-ZQxxws 2024/11/23-15:40:08.729775 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst): first pass time = 0.90's, 149.655'MB/sec 2024/11/23-15:40:10.072261 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst): old prealloc_size = 147639500, real_size = 34156096 2024/11/23-15:40:10.072804 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0322 GB avg-val = 116.16 avg-zval = 34.53 2024/11/23-15:40:10.073627 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:10.918092 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267133.sst): second pass time = 1.34's, 80.635'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.152'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 305.002'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36085.010'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.635'MB/sec zip pipeline throughput = 80.635'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.16 avg-zval = 34.53 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.16 avg-zval = 34.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5142 value = 3.3637 dict = 1.00 all = 3.4969 } Zip/UnZip{ index = 0.0951 value = 0.2973 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 10.784161 GB avg = 0.158 KB (by entry num) total key len = 1.887702 GB avg = 0.028 KB total ukey len = 1.317079 GB avg = 0.016 KB total ukey num = 0.081510473 Billion total entry num = 0.068318202 Billion write speed all = 0.521503689 MB/sec (with seq num) write speed all = 0.499010900 MB/sec (without seq num) 2024/11/23-15:40:10.969937 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267133: 931033 keys, 38382392 bytes, temperature: kWarm 2024/11/23-15:40:10.970023 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347610969973, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267133, "file_size": 38382392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32152176, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148902, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347607, "oldest_key_time": 0, "file_creation_time": 1732347607, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267133, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:11.008445 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst) /dev/shm/ToplingTemp/Topling-98436-lh2lir 2024/11/23-15:40:11.999328 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst): first pass time = 0.99's, 135.458'MB/sec 2024/11/23-15:40:13.303760 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst): old prealloc_size = 147639500, real_size = 32061872 2024/11/23-15:40:13.304308 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928021 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.63 avg-zval = 32.40 2024/11/23-15:40:13.305138 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:14.138351 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267134.sst): second pass time = 1.30's, 83.060'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.610'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.160'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 288.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26974.395'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.060'MB/sec zip pipeline throughput = 83.060'MB/sec entries = 928021 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 32.40 usrkeys = 928021 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 32.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928021 bytes = 3480096 } seq expand size = 6496147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5124 value = 3.6001 dict = 1.00 all = 3.6877 } Zip/UnZip{ index = 0.0951 value = 0.2778 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 10.898891 GB avg = 0.157 KB (by entry num) total key len = 1.913687 GB avg = 0.028 KB total ukey len = 1.335639 GB avg = 0.016 KB total ukey num = 0.082438494 Billion total entry num = 0.069246223 Billion write speed all = 0.527224825 MB/sec (with seq num) write speed all = 0.504429519 MB/sec (without seq num) 2024/11/23-15:40:14.183045 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267134: 928021 keys, 36395604 bytes, temperature: kWarm 2024/11/23-15:40:14.183115 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347614183080, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267134, "file_size": 36395604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 30064096, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984588, "raw_average_key_size": 28, "raw_value_size": 108233246, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928021, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347610, "oldest_key_time": 0, "file_creation_time": 1732347610, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267134, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:14.221467 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst) /dev/shm/ToplingTemp/Topling-98436-LpoAut 2024/11/23-15:40:15.230704 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst): first pass time = 1.01's, 132.995'MB/sec 2024/11/23-15:40:16.521868 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst): old prealloc_size = 147639500, real_size = 32695120 2024/11/23-15:40:16.522441 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923208 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.38 avg-zval = 33.24 2024/11/23-15:40:16.523276 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:17.354416 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267135.sst): second pass time = 1.29's, 84.013'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.461'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.149'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 297.684'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36108.002'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.013'MB/sec zip pipeline throughput = 84.013'MB/sec entries = 923208 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.38 avg-zval = 33.24 usrkeys = 923208 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.38 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923208 bytes = 3462048 } seq expand size = 6462456 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3752 value = 3.5317 dict = 1.00 all = 3.6261 } Zip/UnZip{ index = 0.0964 value = 0.2832 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 11.013721 GB avg = 0.157 KB (by entry num) total key len = 1.939537 GB avg = 0.028 KB total ukey len = 1.354103 GB avg = 0.016 KB total ukey num = 0.083361702 Billion total entry num = 0.070169431 Billion write speed all = 0.532943151 MB/sec (with seq num) write speed all = 0.509846989 MB/sec (without seq num) 2024/11/23-15:40:17.399733 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267135: 923208 keys, 37014858 bytes, temperature: kWarm 2024/11/23-15:40:17.399806 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347617399769, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267135, "file_size": 37014858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30684480, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849824, "raw_average_key_size": 28, "raw_value_size": 108368020, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923208, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347614, "oldest_key_time": 0, "file_creation_time": 1732347614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267135, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:17.436440 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst) /dev/shm/ToplingTemp/Topling-98436-7U5iKt 2024/11/23-15:40:18.281639 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst): first pass time = 0.85's, 158.807'MB/sec 2024/11/23-15:40:19.735633 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst): old prealloc_size = 147639500, real_size = 36053728 2024/11/23-15:40:19.736189 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930893 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.18 avg-zval = 36.58 2024/11/23-15:40:19.736454 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:20.462111 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267136.sst): second pass time = 1.45's, 74.423'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 51.951'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.677'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.387'MB/sec write SST data time = 0.12's, 304.705'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31877.037'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.423'MB/sec zip pipeline throughput = 74.423'MB/sec entries = 930893 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 36.58 usrkeys = 930893 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.77 avg-zval = 36.58 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 210889 bytes = 685408 } seq expand size = 1476223 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5126 value = 3.1763 dict = 1.00 all = 3.5563 } Zip/UnZip{ index = 0.0951 value = 0.3148 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 11.123350 GB avg = 0.156 KB (by entry num) total key len = 1.965602 GB avg = 0.028 KB total ukey len = 1.372721 GB avg = 0.016 KB total ukey num = 0.084292595 Billion total entry num = 0.071100324 Billion write speed all = 0.538457240 MB/sec (with seq num) write speed all = 0.515057667 MB/sec (without seq num) 2024/11/23-15:40:20.509699 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267136: 930893 keys, 37740401 bytes, temperature: kWarm 2024/11/23-15:40:20.509769 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347620509733, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267136, "file_size": 37740401, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 34049808, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065004, "raw_average_key_size": 28, "raw_value_size": 108152830, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930893, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347617, "oldest_key_time": 0, "file_creation_time": 1732347617, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267136, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:20.545242 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst) /dev/shm/ToplingTemp/Topling-98436-0lsC2s 2024/11/23-15:40:21.507603 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst): first pass time = 0.96's, 139.472'MB/sec 2024/11/23-15:40:22.968908 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst): old prealloc_size = 147639500, real_size = 37330272 2024/11/23-15:40:22.969476 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925219 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 117.07 avg-zval = 38.18 2024/11/23-15:40:22.969880 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:23.703890 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267137.sst): second pass time = 1.46's, 74.159'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.291'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.379'MB/sec write SST data time = 0.12's, 320.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34920.229'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.159'MB/sec zip pipeline throughput = 74.159'MB/sec entries = 925219 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.07 avg-zval = 38.18 usrkeys = 925219 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.67 avg-zval = 38.18 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 343630 bytes = 1288624 } seq expand size = 2405410 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4466 value = 3.0659 dict = 1.00 all = 3.3874 } Zip/UnZip{ index = 0.0957 value = 0.3262 dict = 1.00 all = 0.2952 } ---------------------------- total value len = 11.234067 GB avg = 0.156 KB (by entry num) total key len = 1.991508 GB avg = 0.028 KB total ukey len = 1.391226 GB avg = 0.016 KB total ukey num = 0.085217814 Billion total entry num = 0.072025543 Billion write speed all = 0.544005136 MB/sec (with seq num) write speed all = 0.520304229 MB/sec (without seq num) 2024/11/23-15:40:23.751530 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267137: 925219 keys, 39622847 bytes, temperature: kWarm 2024/11/23-15:40:23.751609 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347623751563, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267137, "file_size": 39622847, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4049532967, "table_properties": {"data_size": 35327440, "index_size": 1771328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906132, "raw_average_key_size": 28, "raw_value_size": 108311634, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925219, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347620, "oldest_key_time": 0, "file_creation_time": 1732347620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267137, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:23.790239 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst) /dev/shm/ToplingTemp/Topling-98436-SyO1wr 2024/11/23-15:40:24.648172 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst): first pass time = 0.86's, 156.450'MB/sec 2024/11/23-15:40:25.989062 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst): old prealloc_size = 147639500, real_size = 37273984 2024/11/23-15:40:25.989587 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927123 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 116.77 avg-zval = 38.04 2024/11/23-15:40:25.990315 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:26.792189 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267138.sst): second pass time = 1.34's, 80.783'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 51.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3450.287'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 328.258'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31396.083'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.783'MB/sec zip pipeline throughput = 80.783'MB/sec entries = 927123 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.77 avg-zval = 38.04 usrkeys = 927123 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.77 avg-zval = 38.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927123 bytes = 3013168 } seq expand size = 6489861 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.4432 value = 3.0697 dict = 1.00 all = 3.2621 } Zip/UnZip{ index = 0.0958 value = 0.3258 dict = 1.00 all = 0.3066 } ---------------------------- total value len = 11.348816 GB avg = 0.156 KB (by entry num) total key len = 2.017467 GB avg = 0.028 KB total ukey len = 1.409768 GB avg = 0.016 KB total ukey num = 0.086144937 Billion total entry num = 0.072952666 Billion write speed all = 0.549723008 MB/sec (with seq num) write speed all = 0.525720069 MB/sec (without seq num) 2024/11/23-15:40:26.842481 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267138: 927123 keys, 41145186 bytes, temperature: kWarm 2024/11/23-15:40:26.842554 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347626842517, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267138, "file_size": 41145186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4042068720, "largest_seqno": 4075878979, "table_properties": {"data_size": 35266480, "index_size": 1775552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959444, "raw_average_key_size": 28, "raw_value_size": 108258402, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927123, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347623, "oldest_key_time": 0, "file_creation_time": 1732347623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267138, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:26.884149 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst) /dev/shm/ToplingTemp/Topling-98436-xCdK6r 2024/11/23-15:40:27.892270 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst): first pass time = 1.01's, 133.142'MB/sec 2024/11/23-15:40:29.207802 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst): old prealloc_size = 147639500, real_size = 35312208 2024/11/23-15:40:29.208360 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929310 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.43 avg-zval = 35.82 2024/11/23-15:40:29.210199 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:40:30.036704 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267139.sst): second pass time = 1.31's, 82.291'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3318.803'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 306.059'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36487.811'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.291'MB/sec zip pipeline throughput = 82.291'MB/sec entries = 929310 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.43 avg-zval = 35.82 usrkeys = 929310 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.43 avg-zval = 35.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929310 bytes = 2904112 } seq expand size = 6505170 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.3766 value = 3.2503 dict = 1.00 all = 3.4354 } Zip/UnZip{ index = 0.0964 value = 0.3077 dict = 1.00 all = 0.2911 } ---------------------------- total value len = 11.463518 GB avg = 0.155 KB (by entry num) total key len = 2.043488 GB avg = 0.028 KB total ukey len = 1.428354 GB avg = 0.016 KB total ukey num = 0.087074247 Billion total entry num = 0.073881976 Billion write speed all = 0.555436488 MB/sec (with seq num) write speed all = 0.531131030 MB/sec (without seq num) 2024/11/23-15:40:30.085012 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267139: 929310 keys, 39069040 bytes, temperature: kWarm 2024/11/23-15:40:30.085083 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347630085047, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267139, "file_size": 39069040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4048925223, "largest_seqno": 4075878833, "table_properties": {"data_size": 33288512, "index_size": 1791168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020680, "raw_average_key_size": 28, "raw_value_size": 108197124, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929310, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347626, "oldest_key_time": 0, "file_creation_time": 1732347626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267139, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:30.123392 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst) /dev/shm/ToplingTemp/Topling-98436-rzy6zs 2024/11/23-15:40:30.285310 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst): first pass time = 0.16's, 247.614'MB/sec 2024/11/23-15:40:30.670654 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst): old prealloc_size = 147639500, real_size = 11295888 2024/11/23-15:40:30.670871 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 277182 prefix = 4 raw-key = 0.0044 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0323 GB zip-val = 0.0107 GB avg-val = 116.61 avg-zval = 38.60 2024/11/23-15:40:30.671137 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.040 GB 2024/11/23-15:40:30.916055 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00/267140.sst): second pass time = 0.38's, 84.031'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 50.637'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2739.520'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.336'MB/sec write SST data time = 0.04's, 319.715'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.330'MB, throughput = 10817.887'MB/sec zip my value time = 0.38's, unzip length = 0.032'GB zip my value throughput = 84.031'MB/sec zip pipeline throughput = 84.031'MB/sec entries = 277182 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.61 avg-zval = 38.60 usrkeys = 277182 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.61 avg-zval = 38.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 277182 bytes = 866208 } seq expand size = 1940274 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0055 GB value = 0.0323 GB dict = 0.33 MB all = 0.0401 GB } __ZipSize{ index = 0.0005 GB value = 0.0107 GB dict = 0.33 MB all = 0.0124 GB } UnZip/Zip{ index = 10.4904 value = 3.0214 dict = 1.00 all = 3.2260 } Zip/UnZip{ index = 0.0953 value = 0.3310 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 11.497781 GB avg = 0.155 KB (by entry num) total key len = 2.051249 GB avg = 0.028 KB total ukey len = 1.433898 GB avg = 0.016 KB total ukey num = 0.087351429 Billion total entry num = 0.074159158 Billion write speed all = 0.557144458 MB/sec (with seq num) write speed all = 0.532748695 MB/sec (without seq num) 2024/11/23-15:40:30.936161 139957592835648 [compaction_job.cc:1948] [default] [JOB 29569] Generated table #267140: 277182 keys, 12425353 bytes, temperature: kWarm 2024/11/23-15:40:30.936228 139957592835648 EVENT_LOG_v1 {"time_micros": 1732347630936192, "cf_name": "default", "job": 29569, "event": "table_file_creation", "file_number": 267140, "file_size": 12425353, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4045620422, "largest_seqno": 4075878708, "table_properties": {"data_size": 10697984, "index_size": 528448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7761096, "raw_average_key_size": 28, "raw_value_size": 32322656, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 277182, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732347630, "oldest_key_time": 0, "file_creation_time": 1732347630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 267140, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:40:30.950435 139957592835648 [compaction_job.cc:703] job-29569: subcompact[0], size: 0.873371 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-29569/att-00 [267117,267118,267119,267120,267121,267122,267123,267124,267125,267126,267127,267128,267129,267130,267131,267132,267133,267134,267135,267136,267137,267138,267139,267140] 2024/11/23-15:40:31.090422 139957592835648 (Original Log Time 2024/11/23-15:39:14.014435) EVENT_LOG_v1 {"time_micros": 1732347554014389, "job": 29569, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [267082], "files_L3": [267106, 267107, 267029, 267030, 267031, 267032, 267033, 267034, 267035, 267036, 267037, 267038, 267039, 267040, 267041, 267042, 267043, 267044, 267045, 267046, 267047, 267048, 267056, 267108], "score": 0.997978, "input_data_size": 896282318, "oldest_snapshot_seqno": 4076186639} 2024/11/23-15:40:31.090426 139957592835648 (Original Log Time 2024/11/23-15:40:31.088816) [compaction_job.cc:2022] [default] [JOB 29569] Compacted 1@2 + 24@3 files to L3 => 873371082 bytes 2024/11/23-15:40:31.090427 139957592835648 (Original Log Time 2024/11/23-15:40:31.090339) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 11.6 rd, 11.4 wr, level 3, files in(1, 24) out(24 +0 blob) MB in(27.1, 827.6 +0.0 blob) out(832.9 +0.0 blob), read-write-amplify(62.2) write-amplify(30.7) OK, records in: 21843305, records dropped: 42311 output_compression: Snappy 2024/11/23-15:40:31.090430 139957592835648 (Original Log Time 2024/11/23-15:40:31.090393) EVENT_LOG_v1 {"time_micros": 1732347631090369, "cf": "default", "job": 29569, "event": "compaction_finished", "compaction_time_micros": 76936008, "compaction_time_cpu_micros": 32475190, "output_level": 3, "num_output_files": 24, "total_output_size": 873371082, "num_input_records": 21843305, "num_output_records": 21800994, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 593437278, "file_range_sync_nanos": 0, "file_fsync_nanos": 17057, "file_prepare_write_nanos": 65461, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/23-15:40:31.223000 139957592835648 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732347553,"executesMs":77228,"compactionJobId":29569,"attempt":0,"compactionInputRawBytes":3133100995,"compactionInputZipBytes":896282318,"compactionOutputRawBytes":0,"compactionOutputZipBytes":873371082,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c8d0d0dff69a101b7e1eb9ac5646055e-f5c770739c787567-00"}}, response = 2024/11/23-15:40:31.223080 139957592835648 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-29569/att-00: olev 3, work 77.229 s, result 7.713 ms, install 1.630 ms, input{raw 2.918 GiB zip 854.761 MiB}