2024/11/23-18:36:07.417074 139957206771264 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00: fork to child time = 0.007715 sec 2024/11/23-18:36:07.417878 139957206771264 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00 2024/11/23-18:36:07.425334 139957206771264 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00 2024/11/23-18:36:07.426215 139957206771264 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:36:07.426474 139957206771264 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:36:07.479573 139957206771264 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33265/att-00/MANIFEST-206347 2024/11/23-18:36:07.511496 139957206771264 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33265/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 275185, last_sequence is 4187845614, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 275012 2024/11/23-18:36:07.511503 139957206771264 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:36:07.511624 139957206771264 [version_set.cc:5721] Creating manifest 275185 2024/11/23-18:36:07.547154 139957206771264 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:36:07.567097 139957206771264 [compaction_job.cc:2437] [default] [JOB 33265] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-18:36:07.567111 139957206771264 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24907 Base level 2, inputs: [275151(27MB)], [275174(37MB) 275175(4449KB) 275100(38MB) 275101(38MB) 275102(36MB) 275103(37MB) 275104(38MB) 275105(35MB) 275106(34MB) 275107(37MB) 275108(33MB) 275109(37MB) 275110(35MB) 275111(34MB) 275112(36MB) 275113(37MB) 275114(36MB) 275115(37MB) 275116(36MB) 275179(35MB)] 2024/11/23-18:36:07.573791 139957206771264 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:36:07.573820 139957206771264 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:36:07.573839 139957206771264 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:36:07.573844 139957206771264 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:36:07.573855 139957206771264 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:36:08.302856 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst) /dev/shm/ToplingTemp/Topling-152230-nc28A7 2024/11/23-18:36:08.869196 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst): first pass time = 0.57's, 237.007'MB/sec 2024/11/23-18:36:10.777847 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst): old prealloc_size = 147639500, real_size = 34923216 2024/11/23-18:36:10.778440 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928740 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0329 GB avg-val = 116.52 avg-zval = 35.44 2024/11/23-18:36:10.779210 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:11.656322 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275186.sst): second pass time = 1.91's, 56.737'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.164'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3103.392'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.15's, 251.576'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36341.199'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.737'MB/sec zip pipeline throughput = 125.751'MB/sec entries = 928740 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 35.44 usrkeys = 928740 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 35.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928740 bytes = 3018416 } seq expand size = 6501180 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4467 value = 3.2879 dict = 1.00 all = 3.4597 } Zip/UnZip{ index = 0.0957 value = 0.3041 dict = 1.00 all = 0.2890 } ---------------------------- total value len = 8.963334 GB avg = 0.168 KB (by entry num) total key len = 1.466826 GB avg = 0.028 KB total ukey len = 1.016453 GB avg = 0.015 KB total ukey num = 0.066479175 Billion total entry num = 0.053286904 Billion write speed all = 0.299206164 MB/sec (with seq num) write speed all = 0.286977189 MB/sec (without seq num) 2024/11/23-18:36:11.709623 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275186: 928740 keys, 38794868 bytes, temperature: kWarm 2024/11/23-18:36:11.709749 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358171709669, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275186, "file_size": 38794868, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4131604997, "largest_seqno": 4187591746, "table_properties": {"data_size": 32912768, "index_size": 1778048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004720, "raw_average_key_size": 28, "raw_value_size": 108213044, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358168, "oldest_key_time": 0, "file_creation_time": 1732358168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:11.789080 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst) /dev/shm/ToplingTemp/Topling-152230-FdE2J7 2024/11/23-18:36:13.016082 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst): first pass time = 1.23's, 109.396'MB/sec 2024/11/23-18:36:14.743292 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst): old prealloc_size = 147639500, real_size = 36514032 2024/11/23-18:36:14.743850 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930895 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0345 GB avg-val = 116.18 avg-zval = 37.07 2024/11/23-18:36:14.744717 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:15.783965 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275187.sst): second pass time = 1.73's, 62.646'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 39.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.686'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.276'MB/sec write SST data time = 0.20's, 206.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37488.008'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.646'MB/sec zip pipeline throughput = 62.646'MB/sec entries = 930895 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 37.07 usrkeys = 930895 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 37.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930895 bytes = 3490864 } seq expand size = 6516265 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.5130 value = 3.1339 dict = 1.00 all = 3.2848 } Zip/UnZip{ index = 0.0951 value = 0.3191 dict = 1.00 all = 0.3044 } ---------------------------- total value len = 9.078003 GB avg = 0.167 KB (by entry num) total key len = 1.492891 GB avg = 0.028 KB total ukey len = 1.035071 GB avg = 0.015 KB total ukey num = 0.067410070 Billion total entry num = 0.054217799 Billion write speed all = 0.303207449 MB/sec (with seq num) write speed all = 0.290766314 MB/sec (without seq num) 2024/11/23-18:36:15.833340 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275187: 930895 keys, 40860306 bytes, temperature: kWarm 2024/11/23-18:36:15.833402 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358175833369, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275187, "file_size": 40860306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4187591038, "table_properties": {"data_size": 34510176, "index_size": 1770944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065060, "raw_average_key_size": 28, "raw_value_size": 108152786, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930895, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358171, "oldest_key_time": 0, "file_creation_time": 1732358171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:15.873579 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst) /dev/shm/ToplingTemp/Topling-152230-qEqq46 2024/11/23-18:36:16.782509 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst): first pass time = 0.91's, 147.671'MB/sec 2024/11/23-18:36:18.394634 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst): old prealloc_size = 147639500, real_size = 36929200 2024/11/23-18:36:18.395185 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930940 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0349 GB avg-val = 116.17 avg-zval = 37.49 2024/11/23-18:36:18.395784 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:19.346277 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275188.sst): second pass time = 1.61's, 67.133'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.317'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3384.369'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 306.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32539.318'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.133'MB/sec zip pipeline throughput = 67.133'MB/sec entries = 930940 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.17 avg-zval = 37.49 usrkeys = 930940 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.87 avg-zval = 37.49 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 624148 bytes = 2340576 } seq expand size = 4369036 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0403 GB } UnZip/Zip{ index = 10.3770 value = 3.0987 dict = 1.00 all = 3.3329 } Zip/UnZip{ index = 0.0964 value = 0.3227 dict = 1.00 all = 0.3000 } ---------------------------- total value len = 9.190524 GB avg = 0.167 KB (by entry num) total key len = 1.518957 GB avg = 0.028 KB total ukey len = 1.053690 GB avg = 0.015 KB total ukey num = 0.068341010 Billion total entry num = 0.055148739 Billion write speed all = 0.307151184 MB/sec (with seq num) write speed all = 0.294497722 MB/sec (without seq num) 2024/11/23-18:36:19.428448 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275188: 930940 keys, 40270138 bytes, temperature: kWarm 2024/11/23-18:36:19.428493 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358179428467, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275188, "file_size": 40270138, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 34902048, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066320, "raw_average_key_size": 28, "raw_value_size": 108151480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930940, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358175, "oldest_key_time": 0, "file_creation_time": 1732358175, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:19.476950 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst) /dev/shm/ToplingTemp/Topling-152230-3ZzXe7 2024/11/23-18:36:20.446459 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst): first pass time = 0.97's, 138.445'MB/sec 2024/11/23-18:36:22.028451 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst): old prealloc_size = 147639500, real_size = 34217024 2024/11/23-18:36:22.029016 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 117.04 avg-zval = 34.82 2024/11/23-18:36:22.029791 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:23.138072 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275189.sst): second pass time = 1.58's, 68.492'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 34.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.510'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.236'MB/sec write SST data time = 0.13's, 295.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32762.473'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.492'MB/sec zip pipeline throughput = 68.492'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.82 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5121 value = 3.3610 dict = 1.00 all = 3.4928 } Zip/UnZip{ index = 0.0951 value = 0.2975 dict = 1.00 all = 0.2863 } ---------------------------- total value len = 9.305308 GB avg = 0.166 KB (by entry num) total key len = 1.544869 GB avg = 0.028 KB total ukey len = 1.072198 GB avg = 0.015 KB total ukey num = 0.069266415 Billion total entry num = 0.056074144 Billion write speed all = 0.311152539 MB/sec (with seq num) write speed all = 0.298288149 MB/sec (without seq num) 2024/11/23-18:36:23.184953 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275189: 925405 keys, 38426962 bytes, temperature: kWarm 2024/11/23-18:36:23.185041 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358183185003, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275189, "file_size": 38426962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32224880, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358179, "oldest_key_time": 0, "file_creation_time": 1732358179, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:23.229566 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst) /dev/shm/ToplingTemp/Topling-152230-ZbapL8 2024/11/23-18:36:24.380103 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst): first pass time = 1.15's, 116.660'MB/sec 2024/11/23-18:36:25.841054 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst): old prealloc_size = 147639500, real_size = 35739104 2024/11/23-18:36:25.841779 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0337 GB avg-val = 117.23 avg-zval = 36.49 2024/11/23-18:36:25.842670 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:26.772159 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275190.sst): second pass time = 1.46's, 74.205'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 42.902'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2538.259'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.15's, 260.057'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36140.795'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.205'MB/sec zip pipeline throughput = 74.205'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.23 avg-zval = 36.49 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.23 avg-zval = 36.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.3776 value = 3.2123 dict = 1.00 all = 3.3600 } Zip/UnZip{ index = 0.0964 value = 0.3113 dict = 1.00 all = 0.2976 } ---------------------------- total value len = 9.420118 GB avg = 0.165 KB (by entry num) total key len = 1.570745 GB avg = 0.028 KB total ukey len = 1.090681 GB avg = 0.016 KB total ukey num = 0.070190566 Billion total entry num = 0.056998295 Billion write speed all = 0.315154198 MB/sec (with seq num) write speed all = 0.302079154 MB/sec (without seq num) 2024/11/23-18:36:26.820722 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275190: 924151 keys, 39945646 bytes, temperature: kWarm 2024/11/23-18:36:26.820789 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358186820753, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275190, "file_size": 39945646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33726800, "index_size": 1781056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358183, "oldest_key_time": 0, "file_creation_time": 1732358183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:26.858764 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst) /dev/shm/ToplingTemp/Topling-152230-CwvtI8 2024/11/23-18:36:27.713637 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst): first pass time = 0.85's, 157.011'MB/sec 2024/11/23-18:36:29.071361 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst): old prealloc_size = 147639500, real_size = 35478848 2024/11/23-18:36:29.071894 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.20 avg-zval = 35.95 2024/11/23-18:36:29.072643 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:29.884009 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275191.sst): second pass time = 1.36's, 79.708'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.342'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3480.457'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.342'MB/sec write SST data time = 0.13's, 300.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34429.201'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.708'MB/sec zip pipeline throughput = 79.708'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 35.95 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 35.95 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4464 value = 3.2320 dict = 1.00 all = 3.4166 } Zip/UnZip{ index = 0.0957 value = 0.3094 dict = 1.00 all = 0.2927 } ---------------------------- total value len = 9.533510 GB avg = 0.165 KB (by entry num) total key len = 1.596806 GB avg = 0.028 KB total ukey len = 1.109296 GB avg = 0.016 KB total ukey num = 0.071121311 Billion total entry num = 0.057929040 Billion write speed all = 0.319124399 MB/sec (with seq num) write speed all = 0.305837034 MB/sec (without seq num) 2024/11/23-18:36:29.932817 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275191: 930745 keys, 39283508 bytes, temperature: kWarm 2024/11/23-18:36:29.932889 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358189932852, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275191, "file_size": 39283508, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 33464048, "index_size": 1781952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358186, "oldest_key_time": 0, "file_creation_time": 1732358186, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:29.974292 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst) /dev/shm/ToplingTemp/Topling-152230-BXDYh8 2024/11/23-18:36:31.066527 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst): first pass time = 1.09's, 122.889'MB/sec 2024/11/23-18:36:32.775222 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst): old prealloc_size = 147639500, real_size = 32829424 2024/11/23-18:36:32.775745 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.49 avg-zval = 33.18 2024/11/23-18:36:32.776545 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:33.638381 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275192.sst): second pass time = 1.71's, 63.356'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.925'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3551.112'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 288.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34509.254'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.356'MB/sec zip pipeline throughput = 97.942'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.18 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4452 value = 3.5112 dict = 1.00 all = 3.6226 } Zip/UnZip{ index = 0.0957 value = 0.2848 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 9.648220 GB avg = 0.164 KB (by entry num) total key len = 1.622815 GB avg = 0.028 KB total ukey len = 1.127874 GB avg = 0.016 KB total ukey num = 0.072050216 Billion total entry num = 0.058857945 Billion write speed all = 0.323124295 MB/sec (with seq num) write speed all = 0.309625317 MB/sec (without seq num) 2024/11/23-18:36:33.684734 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275192: 928905 keys, 37049942 bytes, temperature: kWarm 2024/11/23-18:36:33.684802 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358193684766, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275192, "file_size": 37049942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30818400, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358189, "oldest_key_time": 0, "file_creation_time": 1732358189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:33.721009 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst) /dev/shm/ToplingTemp/Topling-152230-bte916 2024/11/23-18:36:34.583313 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst): first pass time = 0.86's, 155.656'MB/sec 2024/11/23-18:36:35.879159 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst): old prealloc_size = 147639500, real_size = 32455936 2024/11/23-18:36:35.879732 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933267 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.81 avg-zval = 32.62 2024/11/23-18:36:35.881222 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:36.743135 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275193.sst): second pass time = 1.29's, 83.500'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.467'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3234.737'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 282.117'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35296.513'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.500'MB/sec zip pipeline throughput = 83.500'MB/sec entries = 933267 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.81 avg-zval = 32.62 usrkeys = 933267 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.81 avg-zval = 32.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933267 bytes = 3383104 } seq expand size = 6532869 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5132 value = 3.5500 dict = 1.00 all = 3.6582 } Zip/UnZip{ index = 0.0951 value = 0.2817 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 9.762840 GB avg = 0.163 KB (by entry num) total key len = 1.648946 GB avg = 0.028 KB total ukey len = 1.146539 GB avg = 0.016 KB total ukey num = 0.072983483 Billion total entry num = 0.059791212 Billion write speed all = 0.327130295 MB/sec (with seq num) write speed all = 0.313418495 MB/sec (without seq num) 2024/11/23-18:36:36.787661 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275193: 933267 keys, 36689796 bytes, temperature: kWarm 2024/11/23-18:36:36.787733 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358196787693, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275193, "file_size": 36689796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30447024, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26131476, "raw_average_key_size": 28, "raw_value_size": 108086306, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933267, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358193, "oldest_key_time": 0, "file_creation_time": 1732358193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:36.823916 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst) /dev/shm/ToplingTemp/Topling-152230-zIF1Q7 2024/11/23-18:36:37.885932 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst): first pass time = 1.06's, 126.385'MB/sec 2024/11/23-18:36:39.233547 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst): old prealloc_size = 147639500, real_size = 34568960 2024/11/23-18:36:39.234140 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932565 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0325 GB avg-val = 115.92 avg-zval = 34.90 2024/11/23-18:36:39.235672 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:40.078192 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275194.sst): second pass time = 1.35's, 80.292'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.075'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3132.707'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 304.102'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 35633.231'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.292'MB/sec zip pipeline throughput = 80.292'MB/sec entries = 932565 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.92 avg-zval = 34.90 usrkeys = 932565 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.92 avg-zval = 34.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932565 bytes = 3497136 } seq expand size = 6527955 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4439 value = 3.3212 dict = 1.00 all = 3.4489 } Zip/UnZip{ index = 0.0957 value = 0.3011 dict = 1.00 all = 0.2899 } ---------------------------- total value len = 9.877474 GB avg = 0.163 KB (by entry num) total key len = 1.675058 GB avg = 0.028 KB total ukey len = 1.165190 GB avg = 0.016 KB total ukey num = 0.073916048 Billion total entry num = 0.060723777 Billion write speed all = 0.331133256 MB/sec (with seq num) write speed all = 0.317208923 MB/sec (without seq num) 2024/11/23-18:36:40.126914 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275194: 932565 keys, 38916178 bytes, temperature: kWarm 2024/11/23-18:36:40.127005 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358200126948, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275194, "file_size": 38916178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32549808, "index_size": 1785856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26111820, "raw_average_key_size": 28, "raw_value_size": 108105982, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932565, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358196, "oldest_key_time": 0, "file_creation_time": 1732358196, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:40.163485 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst) /dev/shm/ToplingTemp/Topling-152230-9V0VY5 2024/11/23-18:36:41.158469 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst): first pass time = 0.99's, 134.899'MB/sec 2024/11/23-18:36:42.434784 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst): old prealloc_size = 147639500, real_size = 30452656 2024/11/23-18:36:42.435349 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929593 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.38 avg-zval = 30.57 2024/11/23-18:36:42.438020 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:43.260763 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275195.sst): second pass time = 1.28's, 84.822'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.121'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.298'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 269.030'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35245.372'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.822'MB/sec zip pipeline throughput = 84.822'MB/sec entries = 929593 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.38 avg-zval = 30.57 usrkeys = 929593 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.38 avg-zval = 30.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929593 bytes = 3485984 } seq expand size = 6507151 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3097 value = 3.8072 dict = 1.00 all = 3.8580 } Zip/UnZip{ index = 0.0970 value = 0.2627 dict = 1.00 all = 0.2592 } ---------------------------- total value len = 9.992170 GB avg = 0.162 KB (by entry num) total key len = 1.701087 GB avg = 0.028 KB total ukey len = 1.183782 GB avg = 0.016 KB total ukey num = 0.074845641 Billion total entry num = 0.061653370 Billion write speed all = 0.335136322 MB/sec (with seq num) write speed all = 0.321000117 MB/sec (without seq num) 2024/11/23-18:36:43.304037 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275195: 929593 keys, 34789684 bytes, temperature: kWarm 2024/11/23-18:36:43.304115 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358203304073, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275195, "file_size": 34789684, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3421586384, "largest_seqno": 4012776300, "table_properties": {"data_size": 28416736, "index_size": 1803328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26028604, "raw_average_key_size": 28, "raw_value_size": 108189246, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929593, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358200, "oldest_key_time": 0, "file_creation_time": 1732358200, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:43.345376 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst) /dev/shm/ToplingTemp/Topling-152230-kRM7e5 2024/11/23-18:36:44.349284 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst): first pass time = 1.00's, 133.701'MB/sec 2024/11/23-18:36:45.718611 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst): old prealloc_size = 147639500, real_size = 35290464 2024/11/23-18:36:45.719192 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924347 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.20 avg-zval = 36.03 2024/11/23-18:36:45.720794 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:46.559071 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275196.sst): second pass time = 1.37's, 79.162'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3156.251'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 307.215'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35897.678'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.162'MB/sec zip pipeline throughput = 79.162'MB/sec entries = 924347 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 36.03 usrkeys = 924347 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 36.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924347 bytes = 3466320 } seq expand size = 6470429 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5131 value = 3.2533 dict = 1.00 all = 3.3882 } Zip/UnZip{ index = 0.0951 value = 0.3074 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 10.106977 GB avg = 0.162 KB (by entry num) total key len = 1.726969 GB avg = 0.028 KB total ukey len = 1.202269 GB avg = 0.016 KB total ukey num = 0.075769988 Billion total entry num = 0.062577717 Billion write speed all = 0.339136480 MB/sec (with seq num) write speed all = 0.324789692 MB/sec (without seq num) 2024/11/23-18:36:46.606402 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275196: 924347 keys, 39613492 bytes, temperature: kWarm 2024/11/23-18:36:46.606471 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358206606435, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275196, "file_size": 39613492, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33300752, "index_size": 1758464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881716, "raw_average_key_size": 28, "raw_value_size": 108336126, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924347, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358203, "oldest_key_time": 0, "file_creation_time": 1732358203, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275196, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:46.642852 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst) /dev/shm/ToplingTemp/Topling-152230-XxMLW6 2024/11/23-18:36:47.659106 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst): first pass time = 1.02's, 132.076'MB/sec 2024/11/23-18:36:48.977092 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst): old prealloc_size = 147639500, real_size = 31569712 2024/11/23-18:36:48.977637 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.15 avg-zval = 31.74 2024/11/23-18:36:48.979202 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:49.807616 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275197.sst): second pass time = 1.32's, 82.108'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3505.277'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 285.417'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34432.284'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.108'MB/sec zip pipeline throughput = 82.108'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 31.74 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 31.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4438 value = 3.6594 dict = 1.00 all = 3.7373 } Zip/UnZip{ index = 0.0958 value = 0.2733 dict = 1.00 all = 0.2676 } ---------------------------- total value len = 10.221642 GB avg = 0.161 KB (by entry num) total key len = 1.753039 GB avg = 0.028 KB total ukey len = 1.220891 GB avg = 0.016 KB total ukey num = 0.076701071 Billion total entry num = 0.063508800 Billion write speed all = 0.343137722 MB/sec (with seq num) write speed all = 0.328578827 MB/sec (without seq num) 2024/11/23-18:36:49.852875 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275197: 931083 keys, 35913000 bytes, temperature: kWarm 2024/11/23-18:36:49.852948 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358209852911, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275197, "file_size": 35913000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29553696, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358206, "oldest_key_time": 0, "file_creation_time": 1732358206, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275197, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:49.890233 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst) /dev/shm/ToplingTemp/Topling-152230-Ji59C6 2024/11/23-18:36:50.917816 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst): first pass time = 1.03's, 130.620'MB/sec 2024/11/23-18:36:52.186418 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst): old prealloc_size = 147639500, real_size = 31606272 2024/11/23-18:36:52.186960 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925492 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0296 GB avg-val = 117.02 avg-zval = 31.96 2024/11/23-18:36:52.188962 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:53.005423 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275198.sst): second pass time = 1.27's, 85.453'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.638'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3531.986'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 273.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35258.546'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.453'MB/sec zip pipeline throughput = 85.453'MB/sec entries = 925492 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.02 avg-zval = 31.96 usrkeys = 925492 avg-key = 20.00 avg-zkey = 1.94 avg-val = 123.55 avg-zval = 31.96 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 862807 bytes = 3127696 } seq expand size = 6039649 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.3085 value = 3.6615 dict = 1.00 all = 3.7556 } Zip/UnZip{ index = 0.0970 value = 0.2731 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 10.335985 GB avg = 0.160 KB (by entry num) total key len = 1.778953 GB avg = 0.028 KB total ukey len = 1.239401 GB avg = 0.016 KB total ukey num = 0.077626563 Billion total entry num = 0.064434292 Billion write speed all = 0.347125030 MB/sec (with seq num) write speed all = 0.332355326 MB/sec (without seq num) 2024/11/23-18:36:53.049777 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275198: 925492 keys, 35738502 bytes, temperature: kWarm 2024/11/23-18:36:53.049846 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358213049811, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275198, "file_size": 35738502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29579120, "index_size": 1795584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913776, "raw_average_key_size": 28, "raw_value_size": 108304060, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925492, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358209, "oldest_key_time": 0, "file_creation_time": 1732358209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:53.090481 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst) /dev/shm/ToplingTemp/Topling-152230-BK2nX8 2024/11/23-18:36:54.091896 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst): first pass time = 1.00's, 134.034'MB/sec 2024/11/23-18:36:55.668592 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst): old prealloc_size = 147639500, real_size = 36779872 2024/11/23-18:36:55.669164 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926490 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.87 avg-zval = 37.55 2024/11/23-18:36:55.669331 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:56.370733 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275199.sst): second pass time = 1.58's, 68.710'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 54.079'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.061'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.401'MB/sec write SST data time = 0.12's, 307.179'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27615.611'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.710'MB/sec zip pipeline throughput = 68.710'MB/sec entries = 926490 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 37.55 usrkeys = 926490 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 37.55 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 62187 bytes = 186576 } seq expand size = 435309 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5138 value = 3.1127 dict = 1.00 all = 3.5349 } Zip/UnZip{ index = 0.0951 value = 0.3213 dict = 1.00 all = 0.2829 } ---------------------------- total value len = 10.444697 GB avg = 0.160 KB (by entry num) total key len = 1.804894 GB avg = 0.028 KB total ukey len = 1.257930 GB avg = 0.016 KB total ukey num = 0.078553053 Billion total entry num = 0.065360782 Billion write speed all = 0.350949358 MB/sec (with seq num) write speed all = 0.335968727 MB/sec (without seq num) 2024/11/23-18:36:56.417095 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275199: 926490 keys, 37969747 bytes, temperature: kWarm 2024/11/23-18:36:56.417175 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358216417138, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275199, "file_size": 37969747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3262239873, "table_properties": {"data_size": 34785616, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941720, "raw_average_key_size": 28, "raw_value_size": 108276092, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926490, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358213, "oldest_key_time": 0, "file_creation_time": 1732358213, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:56.454283 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst) /dev/shm/ToplingTemp/Topling-152230-SYNbK9 2024/11/23-18:36:57.412373 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst): first pass time = 0.96's, 140.094'MB/sec 2024/11/23-18:36:58.811130 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst): old prealloc_size = 147639500, real_size = 35907936 2024/11/23-18:36:58.811661 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926763 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 116.82 avg-zval = 36.59 2024/11/23-18:36:58.812267 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:36:59.598590 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275200.sst): second pass time = 1.40's, 77.451'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 49.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3475.236'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.351'MB/sec write SST data time = 0.13's, 306.594'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30478.031'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.451'MB/sec zip pipeline throughput = 77.451'MB/sec entries = 926763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 36.59 usrkeys = 926763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.99 avg-zval = 36.59 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 552158 bytes = 2070608 } seq expand size = 3865106 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5134 value = 3.1925 dict = 1.00 all = 3.4431 } Zip/UnZip{ index = 0.0951 value = 0.3132 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 10.556830 GB avg = 0.159 KB (by entry num) total key len = 1.830844 GB avg = 0.028 KB total ukey len = 1.276466 GB avg = 0.016 KB total ukey num = 0.079479816 Billion total entry num = 0.066287545 Billion write speed all = 0.354872598 MB/sec (with seq num) write speed all = 0.339680959 MB/sec (without seq num) 2024/11/23-18:36:59.646787 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275200: 926763 keys, 38981314 bytes, temperature: kWarm 2024/11/23-18:36:59.646856 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358219646821, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275200, "file_size": 38981314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4187591810, "table_properties": {"data_size": 33913040, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949364, "raw_average_key_size": 28, "raw_value_size": 108268386, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358216, "oldest_key_time": 0, "file_creation_time": 1732358216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:36:59.684661 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst) /dev/shm/ToplingTemp/Topling-152230-LfMCK6 2024/11/23-18:37:00.740227 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst): first pass time = 1.06's, 127.157'MB/sec 2024/11/23-18:37:02.051075 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst): old prealloc_size = 147639500, real_size = 34628336 2024/11/23-18:37:02.051611 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925690 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.99 avg-zval = 35.24 2024/11/23-18:37:02.053036 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:37:02.901454 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275201.sst): second pass time = 1.31's, 82.666'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3545.229'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 299.882'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31783.779'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.666'MB/sec zip pipeline throughput = 82.666'MB/sec entries = 925690 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.99 avg-zval = 35.24 usrkeys = 925690 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.99 avg-zval = 35.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925690 bytes = 3124224 } seq expand size = 6479830 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4451 value = 3.3196 dict = 1.00 all = 3.4765 } Zip/UnZip{ index = 0.0957 value = 0.3012 dict = 1.00 all = 0.2876 } ---------------------------- total value len = 10.671609 GB avg = 0.159 KB (by entry num) total key len = 1.856763 GB avg = 0.028 KB total ukey len = 1.294979 GB avg = 0.016 KB total ukey num = 0.080405506 Billion total entry num = 0.067213235 Billion write speed all = 0.358869241 MB/sec (with seq num) write speed all = 0.343466912 MB/sec (without seq num) 2024/11/23-18:37:02.947698 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275201: 925690 keys, 38607418 bytes, temperature: kWarm 2024/11/23-18:37:02.947768 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358222947732, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275201, "file_size": 38607418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4187591842, "table_properties": {"data_size": 32624224, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919320, "raw_average_key_size": 28, "raw_value_size": 108298504, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925690, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358219, "oldest_key_time": 0, "file_creation_time": 1732358219, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:37:02.988380 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst) /dev/shm/ToplingTemp/Topling-152230-cX6PW6 2024/11/23-18:37:04.043933 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst): first pass time = 1.06's, 127.158'MB/sec 2024/11/23-18:37:05.387532 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst): old prealloc_size = 147639500, real_size = 34385120 2024/11/23-18:37:05.388096 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929091 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.46 avg-zval = 34.84 2024/11/23-18:37:05.390026 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:37:06.229055 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275202.sst): second pass time = 1.34's, 80.584'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.360'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 301.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33272.408'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.584'MB/sec zip pipeline throughput = 80.584'MB/sec entries = 929091 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.46 avg-zval = 34.84 usrkeys = 929091 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.46 avg-zval = 34.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929091 bytes = 3019568 } seq expand size = 6503637 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4439 value = 3.3423 dict = 1.00 all = 3.5083 } Zip/UnZip{ index = 0.0957 value = 0.2992 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 10.786316 GB avg = 0.158 KB (by entry num) total key len = 1.882778 GB avg = 0.028 KB total ukey len = 1.313561 GB avg = 0.016 KB total ukey num = 0.081334597 Billion total entry num = 0.068142326 Billion write speed all = 0.362865552 MB/sec (with seq num) write speed all = 0.347251805 MB/sec (without seq num) 2024/11/23-18:37:06.276802 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275202: 929091 keys, 38257316 bytes, temperature: kWarm 2024/11/23-18:37:06.276873 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358226276837, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275202, "file_size": 38257316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136176937, "largest_seqno": 4187591880, "table_properties": {"data_size": 32373456, "index_size": 1779200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26014548, "raw_average_key_size": 28, "raw_value_size": 108203274, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929091, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358222, "oldest_key_time": 0, "file_creation_time": 1732358222, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:37:06.313164 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst) /dev/shm/ToplingTemp/Topling-152230-PMVBE8 2024/11/23-18:37:07.160971 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst): first pass time = 0.85's, 158.318'MB/sec 2024/11/23-18:37:08.476228 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst): old prealloc_size = 147639500, real_size = 33869248 2024/11/23-18:37:08.476753 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925846 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.97 avg-zval = 34.43 2024/11/23-18:37:08.478732 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:37:09.329314 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275203.sst): second pass time = 1.31's, 82.388'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.369'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.934'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 295.546'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32168.299'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.388'MB/sec zip pipeline throughput = 82.388'MB/sec entries = 925846 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 34.43 usrkeys = 925846 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 34.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925846 bytes = 3124752 } seq expand size = 6480922 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5129 value = 3.3973 dict = 1.00 all = 3.5461 } Zip/UnZip{ index = 0.0951 value = 0.2943 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 10.901091 GB avg = 0.158 KB (by entry num) total key len = 1.908701 GB avg = 0.028 KB total ukey len = 1.332078 GB avg = 0.016 KB total ukey num = 0.082260443 Billion total entry num = 0.069068172 Billion write speed all = 0.366862840 MB/sec (with seq num) write speed all = 0.351038354 MB/sec (without seq num) 2024/11/23-18:37:09.381496 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275203: 925846 keys, 37849094 bytes, temperature: kWarm 2024/11/23-18:37:09.381568 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358229381531, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275203, "file_size": 37849094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4187591908, "table_properties": {"data_size": 31876272, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923688, "raw_average_key_size": 28, "raw_value_size": 108294108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925846, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358226, "oldest_key_time": 0, "file_creation_time": 1732358226, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:37:09.419290 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst) /dev/shm/ToplingTemp/Topling-152230-Tc0vF9 2024/11/23-18:37:10.387202 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst): first pass time = 0.97's, 138.673'MB/sec 2024/11/23-18:37:11.701935 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst): old prealloc_size = 147639500, real_size = 33956016 2024/11/23-18:37:11.702511 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935534 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0319 GB avg-val = 115.47 avg-zval = 34.13 2024/11/23-18:37:11.703283 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:37:12.540036 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275204.sst): second pass time = 1.31's, 82.214'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.163'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3235.547'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 294.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 37299.726'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.214'MB/sec zip pipeline throughput = 82.214'MB/sec entries = 935534 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.47 avg-zval = 34.13 usrkeys = 935534 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.47 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935534 bytes = 3040496 } seq expand size = 6548738 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4453 value = 3.3830 dict = 1.00 all = 3.5461 } Zip/UnZip{ index = 0.0957 value = 0.2956 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 11.015662 GB avg = 0.157 KB (by entry num) total key len = 1.934896 GB avg = 0.028 KB total ukey len = 1.350789 GB avg = 0.016 KB total ukey num = 0.083195977 Billion total entry num = 0.070003706 Billion write speed all = 0.370860186 MB/sec (with seq num) write speed all = 0.354822831 MB/sec (without seq num) 2024/11/23-18:37:12.588147 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275204: 935534 keys, 37849568 bytes, temperature: kWarm 2024/11/23-18:37:12.588236 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358232588190, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275204, "file_size": 37849568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4137892167, "largest_seqno": 4187591935, "table_properties": {"data_size": 31930656, "index_size": 1791296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26194952, "raw_average_key_size": 28, "raw_value_size": 108022836, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935534, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358229, "oldest_key_time": 0, "file_creation_time": 1732358229, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:37:12.626751 139957206771264 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst) /dev/shm/ToplingTemp/Topling-152230-hUkQk8 2024/11/23-18:37:12.777501 139957206771264 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst): first pass time = 0.15's, 238.665'MB/sec 2024/11/23-18:37:13.124478 139957206771264 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst): old prealloc_size = 147639500, real_size = 9315408 2024/11/23-18:37:13.124691 139957206771264 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 252875 prefix = 4 raw-key = 0.0040 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0289 GB zip-val = 0.0088 GB avg-val = 114.24 avg-zval = 34.68 2024/11/23-18:37:13.124937 139957206771264 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.036 GB 2024/11/23-18:37:13.352101 139957206771264 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00/275205.sst): second pass time = 0.35's, 83.419'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 45.567'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2551.441'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.329'MB/sec write SST data time = 0.04's, 294.324'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.294'MB, throughput = 9198.988'MB/sec zip my value time = 0.35's, unzip length = 0.029'GB zip my value throughput = 83.419'MB/sec zip pipeline throughput = 83.419'MB/sec entries = 252875 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.24 avg-zval = 34.68 usrkeys = 252875 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.24 avg-zval = 34.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 252875 bytes = 821856 } seq expand size = 1770125 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0051 GB value = 0.0289 GB dict = 0.29 MB all = 0.0360 GB } __ZipSize{ index = 0.0005 GB value = 0.0088 GB dict = 0.29 MB all = 0.0104 GB } UnZip/Zip{ index = 10.4861 value = 3.2941 dict = 1.00 all = 3.4683 } Zip/UnZip{ index = 0.0954 value = 0.3036 dict = 1.00 all = 0.2883 } ---------------------------- total value len = 11.046321 GB avg = 0.157 KB (by entry num) total key len = 1.941977 GB avg = 0.028 KB total ukey len = 1.355846 GB avg = 0.016 KB total ukey num = 0.083448852 Billion total entry num = 0.070256581 Billion write speed all = 0.371932263 MB/sec (with seq num) write speed all = 0.355837351 MB/sec (without seq num) 2024/11/23-18:37:13.370634 139957206771264 [compaction_job.cc:1948] [default] [JOB 33265] Generated table #275205: 252875 keys, 10370927 bytes, temperature: kWarm 2024/11/23-18:37:13.370703 139957206771264 EVENT_LOG_v1 {"time_micros": 1732358233370665, "cf_name": "default", "job": 33265, "event": "table_file_creation", "file_number": 275205, "file_size": 10370927, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223986, "largest_seqno": 4187591010, "table_properties": {"data_size": 8769728, "index_size": 482304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7080500, "raw_average_key_size": 28, "raw_value_size": 28888698, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 252875, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358232, "oldest_key_time": 0, "file_creation_time": 1732358232, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:37:13.383085 139957206771264 [compaction_job.cc:703] job-33265: subcompact[0], size: 0.736177 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33265/att-00 [275186,275187,275188,275189,275190,275191,275192,275193,275194,275195,275196,275197,275198,275199,275200,275201,275202,275203,275204,275205] 2024/11/23-18:37:13.504467 139957206771264 (Original Log Time 2024/11/23-18:36:07.567161) EVENT_LOG_v1 {"time_micros": 1732358167567129, "job": 33265, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [275151], "files_L3": [275174, 275175, 275100, 275101, 275102, 275103, 275104, 275105, 275106, 275107, 275108, 275109, 275110, 275111, 275112, 275113, 275114, 275115, 275116, 275179], "score": 0.993249, "input_data_size": 762373342, "oldest_snapshot_seqno": 4187845567} 2024/11/23-18:37:13.504471 139957206771264 (Original Log Time 2024/11/23-18:37:13.502854) [compaction_job.cc:2022] [default] [JOB 33265] Compacted 1@2 + 20@3 files to L3 => 736177406 bytes 2024/11/23-18:37:13.504473 139957206771264 (Original Log Time 2024/11/23-18:37:13.504395) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.6 rd, 11.2 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 699.9 +0.0 blob) out(702.1 +0.0 blob), read-write-amplify(52.7) write-amplify(25.9) OK, records in: 17977653, records dropped: 79236 output_compression: Snappy 2024/11/23-18:37:13.504476 139957206771264 (Original Log Time 2024/11/23-18:37:13.504440) EVENT_LOG_v1 {"time_micros": 1732358233504419, "cf": "default", "job": 33265, "event": "compaction_finished", "compaction_time_micros": 65815931, "compaction_time_cpu_micros": 27510293, "output_level": 3, "num_output_files": 20, "total_output_size": 736177406, "num_input_records": 17977653, "num_output_records": 17898417, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 531573710, "file_range_sync_nanos": 0, "file_fsync_nanos": 14576, "file_prepare_write_nanos": 60612, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:37:13.637119 139957206771264 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732358167,"executesMs":66077,"compactionJobId":33265,"attempt":0,"compactionInputRawBytes":2597359086,"compactionInputZipBytes":762373342,"compactionOutputRawBytes":0,"compactionOutputZipBytes":736177406,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c12ed0eb503fa1e84dd1586fd55d016c-85123bfa1838b9b2-00"}}, response = 2024/11/23-18:37:13.637223 139957206771264 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33265/att-00: olev 3, work 66.078 s, result 7.780 ms, install 1.638 ms, input{raw 2.419 GiB zip 727.056 MiB}