2024/11/24-02:03:46.850704 139957542479424 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00: fork to child time = 0.007467 sec 2024/11/24-02:03:46.851398 139957542479424 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00 2024/11/24-02:03:46.857210 139957542479424 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00 2024/11/24-02:03:46.858112 139957542479424 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-02:03:46.858367 139957542479424 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-02:03:46.920587 139957542479424 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39788/att-00/MANIFEST-206347 2024/11/24-02:03:46.946470 139957542479424 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39788/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 288172, last_sequence is 4386209782, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287990 2024/11/24-02:03:46.946481 139957542479424 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-02:03:46.946719 139957542479424 [version_set.cc:5721] Creating manifest 288172 2024/11/24-02:03:46.986910 139957542479424 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-02:03:47.008529 139957542479424 [compaction_job.cc:2437] [default] [JOB 39788] Compacting 1@2 + 16@3 files to L3, score 0.79, subcompactions 1 : 1 2024/11/24-02:03:47.008543 139957542479424 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29812 Base level 2, inputs: [288121(27MB)], [288163(34MB) 288164(976KB) 288071(35MB) 288072(35MB) 288073(35MB) 288074(35MB) 288075(35MB) 288076(35MB) 288077(35MB) 288078(35MB) 288079(30MB) 288080(35MB) 288081(30MB) 288082(33MB) 288083(34MB) 288165(32MB)] 2024/11/24-02:03:47.015006 139957542479424 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:03:47.015035 139957542479424 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:03:47.015054 139957542479424 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-02:03:47.015060 139957542479424 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-02:03:47.015077 139957542479424 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-02:03:47.728665 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst) /dev/shm/ToplingTemp/Topling-243955-4fvZxV 2024/11/24-02:03:48.273342 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst): first pass time = 0.54's, 246.438'MB/sec 2024/11/24-02:03:49.594471 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst): old prealloc_size = 147639500, real_size = 32521744 2024/11/24-02:03:49.595019 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924294 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.21 avg-zval = 33.03 2024/11/24-02:03:49.595774 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:03:50.424090 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288173.sst): second pass time = 1.32's, 82.055'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.995'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3336.103'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.12's, 292.304'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33928.175'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.055'MB/sec zip pipeline throughput = 82.055'MB/sec entries = 924294 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 33.03 usrkeys = 924294 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924294 bytes = 3119504 } seq expand size = 6470058 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5110 value = 3.5484 dict = 1.00 all = 3.6774 } Zip/UnZip{ index = 0.0951 value = 0.2818 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 8.963427 GB avg = 0.168 KB (by entry num) total key len = 1.466701 GB avg = 0.028 KB total ukey len = 1.016364 GB avg = 0.015 KB total ukey num = 0.066474729 Billion total entry num = 0.053282458 Billion write speed all = 0.168995970 MB/sec (with seq num) write speed all = 0.162089424 MB/sec (without seq num) 2024/11/24-02:03:50.467961 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288173: 924294 keys, 36497748 bytes, temperature: kWarm 2024/11/24-02:03:50.468083 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385030468013, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288173, "file_size": 36497748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4385745755, "table_properties": {"data_size": 30531776, "index_size": 1758720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880232, "raw_average_key_size": 28, "raw_value_size": 108337512, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924294, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385027, "oldest_key_time": 0, "file_creation_time": 1732385027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288173, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:03:50.505756 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst) /dev/shm/ToplingTemp/Topling-243955-57AkJT 2024/11/24-02:03:51.451484 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst): first pass time = 0.95's, 141.925'MB/sec 2024/11/24-02:03:52.760954 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst): old prealloc_size = 147639500, real_size = 32936064 2024/11/24-02:03:52.761515 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.16 avg-zval = 33.45 2024/11/24-02:03:52.762280 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:03:53.607636 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288174.sst): second pass time = 1.31's, 82.777'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.598'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.754'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 291.694'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31036.003'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.777'MB/sec zip pipeline throughput = 82.777'MB/sec entries = 924650 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.16 avg-zval = 33.45 usrkeys = 924650 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.16 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924650 bytes = 3120704 } seq expand size = 6472550 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4436 value = 3.5019 dict = 1.00 all = 3.6361 } Zip/UnZip{ index = 0.0958 value = 0.2856 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 9.078227 GB avg = 0.167 KB (by entry num) total key len = 1.492592 GB avg = 0.028 KB total ukey len = 1.034857 GB avg = 0.015 KB total ukey num = 0.067399379 Billion total entry num = 0.054207108 Billion write speed all = 0.171266695 MB/sec (with seq num) write speed all = 0.164240657 MB/sec (without seq num) 2024/11/24-02:03:53.653438 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288174: 924650 keys, 36912296 bytes, temperature: kWarm 2024/11/24-02:03:53.653510 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385033653473, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288174, "file_size": 36912296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4385746105, "table_properties": {"data_size": 30933936, "index_size": 1770752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890200, "raw_average_key_size": 28, "raw_value_size": 108327568, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385030, "oldest_key_time": 0, "file_creation_time": 1732385030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288174, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:03:53.696131 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst) /dev/shm/ToplingTemp/Topling-243955-kTaN9V 2024/11/24-02:03:54.580876 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst): first pass time = 0.88's, 151.709'MB/sec 2024/11/24-02:03:55.927635 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst): old prealloc_size = 147639500, real_size = 33058928 2024/11/24-02:03:55.928191 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926093 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.93 avg-zval = 33.54 2024/11/24-02:03:55.928958 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:03:56.776493 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288175.sst): second pass time = 1.35's, 80.493'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.635'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3260.833'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 289.247'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31212.239'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.493'MB/sec zip pipeline throughput = 80.493'MB/sec entries = 926093 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.54 usrkeys = 926093 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926093 bytes = 3125584 } seq expand size = 6482651 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5131 value = 3.4858 dict = 1.00 all = 3.6237 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 9.192997 GB avg = 0.167 KB (by entry num) total key len = 1.518522 GB avg = 0.028 KB total ukey len = 1.053379 GB avg = 0.015 KB total ukey num = 0.068325472 Billion total entry num = 0.055133201 Billion write speed all = 0.173537392 MB/sec (with seq num) write speed all = 0.166391685 MB/sec (without seq num) 2024/11/24-02:03:56.821546 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288175: 926093 keys, 37039182 bytes, temperature: kWarm 2024/11/24-02:03:56.821620 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385036821583, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288175, "file_size": 37039182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4385746287, "table_properties": {"data_size": 31065440, "index_size": 1761792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930604, "raw_average_key_size": 28, "raw_value_size": 108287222, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926093, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385033, "oldest_key_time": 0, "file_creation_time": 1732385033, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288175, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:03:56.864508 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst) /dev/shm/ToplingTemp/Topling-243955-HWOCBT 2024/11/24-02:03:57.705356 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst): first pass time = 0.84's, 159.629'MB/sec 2024/11/24-02:03:59.068204 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst): old prealloc_size = 147639500, real_size = 33423792 2024/11/24-02:03:59.068731 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926086 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.93 avg-zval = 33.94 2024/11/24-02:03:59.069480 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:03:59.918755 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288176.sst): second pass time = 1.36's, 79.506'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3525.844'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 299.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25650.118'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.506'MB/sec zip pipeline throughput = 79.506'MB/sec entries = 926086 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.94 usrkeys = 926086 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926086 bytes = 3125552 } seq expand size = 6482602 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5119 value = 3.4453 dict = 1.00 all = 3.5883 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 9.307767 GB avg = 0.166 KB (by entry num) total key len = 1.544453 GB avg = 0.028 KB total ukey len = 1.071900 GB avg = 0.015 KB total ukey num = 0.069251558 Billion total entry num = 0.056059287 Billion write speed all = 0.175807929 MB/sec (with seq num) write speed all = 0.168542564 MB/sec (without seq num) 2024/11/24-02:03:59.964749 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288176: 926086 keys, 37403970 bytes, temperature: kWarm 2024/11/24-02:03:59.964822 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385039964784, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288176, "file_size": 37403970, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4385745781, "table_properties": {"data_size": 31430112, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930408, "raw_average_key_size": 28, "raw_value_size": 108287360, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926086, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385036, "oldest_key_time": 0, "file_creation_time": 1732385036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288176, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:00.002181 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst) /dev/shm/ToplingTemp/Topling-243955-ZL8cjX 2024/11/24-02:04:00.982254 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst): first pass time = 0.98's, 136.953'MB/sec 2024/11/24-02:04:02.299035 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst): old prealloc_size = 147639500, real_size = 33415440 2024/11/24-02:04:02.299561 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926386 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.88 avg-zval = 33.92 2024/11/24-02:04:02.300333 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:03.155567 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288177.sst): second pass time = 1.32's, 82.281'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.660'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3592.643'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 296.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28643.353'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.281'MB/sec zip pipeline throughput = 82.281'MB/sec entries = 926386 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 33.92 usrkeys = 926386 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926386 bytes = 3126560 } seq expand size = 6484702 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5126 value = 3.4460 dict = 1.00 all = 3.5890 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 9.422531 GB avg = 0.165 KB (by entry num) total key len = 1.570391 GB avg = 0.028 KB total ukey len = 1.090428 GB avg = 0.016 KB total ukey num = 0.070177944 Billion total entry num = 0.056985673 Billion write speed all = 0.178077998 MB/sec (with seq num) write speed all = 0.170692960 MB/sec (without seq num) 2024/11/24-02:04:03.200842 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288177: 926386 keys, 37396610 bytes, temperature: kWarm 2024/11/24-02:04:03.200914 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385043200878, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288177, "file_size": 37396610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4385746053, "table_properties": {"data_size": 31421248, "index_size": 1762432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25938808, "raw_average_key_size": 28, "raw_value_size": 108279044, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926386, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385039, "oldest_key_time": 0, "file_creation_time": 1732385039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288177, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:03.237870 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst) /dev/shm/ToplingTemp/Topling-243955-HyHxjW 2024/11/24-02:04:04.117617 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst): first pass time = 0.88's, 152.570'MB/sec 2024/11/24-02:04:05.477292 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst): old prealloc_size = 147639500, real_size = 33540960 2024/11/24-02:04:05.477802 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926136 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.92 avg-zval = 34.06 2024/11/24-02:04:05.478556 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:06.331773 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288178.sst): second pass time = 1.36's, 79.693'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.913'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3617.920'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 299.415'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33234.332'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.693'MB/sec zip pipeline throughput = 79.693'MB/sec entries = 926136 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 34.06 usrkeys = 926136 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 34.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926136 bytes = 3125728 } seq expand size = 6482952 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5139 value = 3.4325 dict = 1.00 all = 3.5771 } Zip/UnZip{ index = 0.0951 value = 0.2913 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 9.537300 GB avg = 0.165 KB (by entry num) total key len = 1.596323 GB avg = 0.028 KB total ukey len = 1.108951 GB avg = 0.016 KB total ukey num = 0.071104080 Billion total entry num = 0.057911809 Billion write speed all = 0.180347977 MB/sec (with seq num) write speed all = 0.172843303 MB/sec (without seq num) 2024/11/24-02:04:06.376967 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288178: 926136 keys, 37521690 bytes, temperature: kWarm 2024/11/24-02:04:06.377061 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385046377021, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288178, "file_size": 37521690, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4385746315, "table_properties": {"data_size": 31547536, "index_size": 1761728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931808, "raw_average_key_size": 28, "raw_value_size": 108286012, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926136, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385043, "oldest_key_time": 0, "file_creation_time": 1732385043, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288178, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:06.416991 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst) /dev/shm/ToplingTemp/Topling-243955-Bd9kqU 2024/11/24-02:04:07.443088 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst): first pass time = 1.03's, 130.809'MB/sec 2024/11/24-02:04:08.781191 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst): old prealloc_size = 147639500, real_size = 33536384 2024/11/24-02:04:08.781720 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926141 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.92 avg-zval = 34.05 2024/11/24-02:04:08.782473 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:09.634380 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288179.sst): second pass time = 1.34's, 80.979'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.974'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3593.300'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 301.879'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30773.949'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.979'MB/sec zip pipeline throughput = 80.979'MB/sec entries = 926141 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.92 avg-zval = 34.05 usrkeys = 926141 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.92 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926141 bytes = 3125744 } seq expand size = 6482987 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4431 value = 3.4343 dict = 1.00 all = 3.5775 } Zip/UnZip{ index = 0.0958 value = 0.2912 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 9.652069 GB avg = 0.164 KB (by entry num) total key len = 1.622255 GB avg = 0.028 KB total ukey len = 1.127474 GB avg = 0.016 KB total ukey num = 0.072030221 Billion total entry num = 0.058837950 Billion write speed all = 0.182617349 MB/sec (with seq num) write speed all = 0.174993065 MB/sec (without seq num) 2024/11/24-02:04:09.679529 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288179: 926141 keys, 37517090 bytes, temperature: kWarm 2024/11/24-02:04:09.679600 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385049679564, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288179, "file_size": 37517090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4385745822, "table_properties": {"data_size": 31530992, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931948, "raw_average_key_size": 28, "raw_value_size": 108285822, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926141, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385046, "oldest_key_time": 0, "file_creation_time": 1732385046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288179, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:09.716192 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst) /dev/shm/ToplingTemp/Topling-243955-HLPrFV 2024/11/24-02:04:10.744878 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst): first pass time = 1.03's, 130.479'MB/sec 2024/11/24-02:04:12.107674 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst): old prealloc_size = 147639500, real_size = 33508688 2024/11/24-02:04:12.108237 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925971 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.95 avg-zval = 34.02 2024/11/24-02:04:12.108954 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:12.958330 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288180.sst): second pass time = 1.36's, 79.529'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.072'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.874'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 300.960'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33142.454'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.529'MB/sec zip pipeline throughput = 79.529'MB/sec entries = 925971 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 34.02 usrkeys = 925971 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 34.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925971 bytes = 3125168 } seq expand size = 6481797 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4460 value = 3.4373 dict = 1.00 all = 3.5802 } Zip/UnZip{ index = 0.0957 value = 0.2909 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 9.766841 GB avg = 0.163 KB (by entry num) total key len = 1.648182 GB avg = 0.028 KB total ukey len = 1.145993 GB avg = 0.016 KB total ukey num = 0.072956192 Billion total entry num = 0.059763921 Billion write speed all = 0.184886395 MB/sec (with seq num) write speed all = 0.177142540 MB/sec (without seq num) 2024/11/24-02:04:13.003478 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288180: 925971 keys, 37488820 bytes, temperature: kWarm 2024/11/24-02:04:13.003554 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385053003516, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288180, "file_size": 37488820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4385745832, "table_properties": {"data_size": 31504128, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927188, "raw_average_key_size": 28, "raw_value_size": 108290602, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925971, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385049, "oldest_key_time": 0, "file_creation_time": 1732385049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288180, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:13.041021 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst) /dev/shm/ToplingTemp/Topling-243955-taGETT 2024/11/24-02:04:14.028847 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst): first pass time = 0.99's, 135.878'MB/sec 2024/11/24-02:04:15.368134 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst): old prealloc_size = 147639500, real_size = 33549264 2024/11/24-02:04:15.368685 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926183 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.92 avg-zval = 34.06 2024/11/24-02:04:15.369424 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:16.212846 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288181.sst): second pass time = 1.34's, 80.929'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.435'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.949'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 300.431'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30241.283'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.929'MB/sec zip pipeline throughput = 80.929'MB/sec entries = 926183 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.92 avg-zval = 34.06 usrkeys = 926183 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.92 avg-zval = 34.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926183 bytes = 3125888 } seq expand size = 6483281 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4443 value = 3.4328 dict = 1.00 all = 3.5759 } Zip/UnZip{ index = 0.0957 value = 0.2913 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 9.881609 GB avg = 0.163 KB (by entry num) total key len = 1.674115 GB avg = 0.028 KB total ukey len = 1.164517 GB avg = 0.016 KB total ukey num = 0.073882375 Billion total entry num = 0.060690104 Billion write speed all = 0.187155432 MB/sec (with seq num) write speed all = 0.179291982 MB/sec (without seq num) 2024/11/24-02:04:16.258269 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288181: 926183 keys, 37533650 bytes, temperature: kWarm 2024/11/24-02:04:16.258340 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385056258305, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288181, "file_size": 37533650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4385746380, "table_properties": {"data_size": 31543936, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933124, "raw_average_key_size": 28, "raw_value_size": 108284742, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926183, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385053, "oldest_key_time": 0, "file_creation_time": 1732385053, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288181, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:16.297779 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst) /dev/shm/ToplingTemp/Topling-243955-4UdFVW 2024/11/24-02:04:17.102044 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst): first pass time = 0.80's, 166.889'MB/sec 2024/11/24-02:04:18.375715 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst): old prealloc_size = 147639500, real_size = 28490224 2024/11/24-02:04:18.376282 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 941822 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0265 GB avg-val = 114.51 avg-zval = 28.09 2024/11/24-02:04:18.377044 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:19.206902 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288182.sst): second pass time = 1.27's, 84.729'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3325.391'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.334'MB/sec write SST data time = 0.13's, 258.945'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 33532.928'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.729'MB/sec zip pipeline throughput = 84.729'MB/sec entries = 941822 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.51 avg-zval = 28.09 usrkeys = 941822 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.51 avg-zval = 28.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 941822 bytes = 3178672 } seq expand size = 6592754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1078 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0265 GB dict = 1.08 MB all = 0.0325 GB } UnZip/Zip{ index = 10.4446 value = 4.0772 dict = 1.00 all = 4.1281 } Zip/UnZip{ index = 0.0957 value = 0.2453 dict = 1.00 all = 0.2422 } ---------------------------- total value len = 9.996049 GB avg = 0.162 KB (by entry num) total key len = 1.700486 GB avg = 0.028 KB total ukey len = 1.183353 GB avg = 0.016 KB total ukey num = 0.074824197 Billion total entry num = 0.061631926 Billion write speed all = 0.189426802 MB/sec (with seq num) write speed all = 0.181441710 MB/sec (without seq num) 2024/11/24-02:04:19.248517 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288182: 941822 keys, 32513360 bytes, temperature: kWarm 2024/11/24-02:04:19.248590 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385059248554, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288182, "file_size": 32513360, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4385745476, "table_properties": {"data_size": 26451104, "index_size": 1803456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26371016, "raw_average_key_size": 28, "raw_value_size": 107846820, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 941822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385056, "oldest_key_time": 0, "file_creation_time": 1732385056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288182, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:19.297500 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst) /dev/shm/ToplingTemp/Topling-243955-J72T3T 2024/11/24-02:04:20.311136 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst): first pass time = 1.01's, 132.417'MB/sec 2024/11/24-02:04:21.655953 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst): old prealloc_size = 147639500, real_size = 33008992 2024/11/24-02:04:21.656531 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927744 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.67 avg-zval = 33.43 2024/11/24-02:04:21.657327 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:22.505484 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288183.sst): second pass time = 1.34's, 80.559'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.548'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3361.340'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 292.400'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35357.606'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.559'MB/sec zip pipeline throughput = 80.559'MB/sec entries = 927744 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 33.43 usrkeys = 927744 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927744 bytes = 3131152 } seq expand size = 6494208 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5116 value = 3.4903 dict = 1.00 all = 3.6281 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 10.110784 GB avg = 0.162 KB (by entry num) total key len = 1.726463 GB avg = 0.028 KB total ukey len = 1.201908 GB avg = 0.016 KB total ukey num = 0.075751941 Billion total entry num = 0.062559670 Billion write speed all = 0.191695408 MB/sec (with seq num) write speed all = 0.183590550 MB/sec (without seq num) 2024/11/24-02:04:22.551577 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288183: 927744 keys, 36993892 bytes, temperature: kWarm 2024/11/24-02:04:22.551651 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385062551615, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288183, "file_size": 36993892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4385745253, "table_properties": {"data_size": 31011664, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976832, "raw_average_key_size": 28, "raw_value_size": 108240940, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927744, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385059, "oldest_key_time": 0, "file_creation_time": 1732385059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288183, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:22.589609 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst) /dev/shm/ToplingTemp/Topling-243955-1E1qIW 2024/11/24-02:04:23.555528 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst): first pass time = 0.97's, 138.958'MB/sec 2024/11/24-02:04:24.817216 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst): old prealloc_size = 147639500, real_size = 28519424 2024/11/24-02:04:24.817780 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 940734 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0265 GB avg-val = 114.67 avg-zval = 28.15 2024/11/24-02:04:24.819672 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:25.645101 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288184.sst): second pass time = 1.26's, 85.558'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.306'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3337.472'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 253.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 35533.358'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.558'MB/sec zip pipeline throughput = 85.558'MB/sec entries = 940734 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.67 avg-zval = 28.15 usrkeys = 940734 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.67 avg-zval = 28.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 940734 bytes = 3174992 } seq expand size = 6585138 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0265 GB dict = 1.08 MB all = 0.0325 GB } UnZip/Zip{ index = 10.4448 value = 4.0735 dict = 1.00 all = 4.1247 } Zip/UnZip{ index = 0.0957 value = 0.2455 dict = 1.00 all = 0.2424 } ---------------------------- total value len = 10.225246 GB avg = 0.161 KB (by entry num) total key len = 1.752804 GB avg = 0.028 KB total ukey len = 1.220723 GB avg = 0.016 KB total ukey num = 0.076692675 Billion total entry num = 0.063500404 Billion write speed all = 0.193965746 MB/sec (with seq num) write speed all = 0.185739430 MB/sec (without seq num) 2024/11/24-02:04:25.685374 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288184: 940734 keys, 32539724 bytes, temperature: kWarm 2024/11/24-02:04:25.685456 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385065685417, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288184, "file_size": 32539724, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4289194921, "largest_seqno": 4385746425, "table_properties": {"data_size": 26482736, "index_size": 1801344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26340552, "raw_average_key_size": 28, "raw_value_size": 107877300, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 940734, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385062, "oldest_key_time": 0, "file_creation_time": 1732385062, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288184, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:25.726407 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst) /dev/shm/ToplingTemp/Topling-243955-T865nU 2024/11/24-02:04:26.720283 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst): first pass time = 0.99's, 135.050'MB/sec 2024/11/24-02:04:28.051906 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst): old prealloc_size = 147639500, real_size = 32493920 2024/11/24-02:04:28.052443 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922527 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.49 avg-zval = 33.07 2024/11/24-02:04:28.053202 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:28.879750 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288185.sst): second pass time = 1.33's, 81.461'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.050'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.315'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 290.056'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33597.220'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.461'MB/sec zip pipeline throughput = 81.461'MB/sec entries = 922527 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.49 avg-zval = 33.07 usrkeys = 922527 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.49 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922527 bytes = 3113536 } seq expand size = 6457689 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5142 value = 3.5527 dict = 1.00 all = 3.6807 } Zip/UnZip{ index = 0.0951 value = 0.2815 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 10.340091 GB avg = 0.161 KB (by entry num) total key len = 1.778635 GB avg = 0.028 KB total ukey len = 1.239173 GB avg = 0.016 KB total ukey num = 0.077615202 Billion total entry num = 0.064422931 Billion write speed all = 0.196233486 MB/sec (with seq num) write speed all = 0.187888096 MB/sec (without seq num) 2024/11/24-02:04:28.925426 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288185: 922527 keys, 36465712 bytes, temperature: kWarm 2024/11/24-02:04:28.925498 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385068925463, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288185, "file_size": 36465712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4385744616, "table_properties": {"data_size": 30508304, "index_size": 1754816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830756, "raw_average_key_size": 28, "raw_value_size": 108387026, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922527, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385065, "oldest_key_time": 0, "file_creation_time": 1732385065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288185, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:28.965622 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst) /dev/shm/ToplingTemp/Topling-243955-4DZS1W 2024/11/24-02:04:29.983098 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst): first pass time = 1.02's, 131.918'MB/sec 2024/11/24-02:04:31.306048 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst): old prealloc_size = 147639500, real_size = 32105600 2024/11/24-02:04:31.306585 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933392 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 115.80 avg-zval = 32.24 2024/11/24-02:04:31.307348 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:32.155682 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288186.sst): second pass time = 1.32's, 81.751'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.498'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3541.331'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 288.378'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32048.395'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.751'MB/sec zip pipeline throughput = 81.751'MB/sec entries = 933392 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 32.24 usrkeys = 933392 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.80 avg-zval = 32.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933392 bytes = 3150208 } seq expand size = 6533744 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5127 value = 3.5912 dict = 1.00 all = 3.7173 } Zip/UnZip{ index = 0.0951 value = 0.2785 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 10.454708 GB avg = 0.160 KB (by entry num) total key len = 1.804770 GB avg = 0.028 KB total ukey len = 1.257841 GB avg = 0.016 KB total ukey num = 0.078548594 Billion total entry num = 0.065356323 Billion write speed all = 0.198502087 MB/sec (with seq num) write speed all = 0.190036234 MB/sec (without seq num) 2024/11/24-02:04:32.203837 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288186: 933392 keys, 36106476 bytes, temperature: kWarm 2024/11/24-02:04:32.203911 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385072203874, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288186, "file_size": 36106476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4385746437, "table_properties": {"data_size": 30096304, "index_size": 1775744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26134976, "raw_average_key_size": 28, "raw_value_size": 108082804, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385068, "oldest_key_time": 0, "file_creation_time": 1732385068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288186, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:32.240851 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst) /dev/shm/ToplingTemp/Topling-243955-KwHv5W 2024/11/24-02:04:33.099624 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst): first pass time = 0.86's, 156.298'MB/sec 2024/11/24-02:04:34.388276 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst): old prealloc_size = 147639500, real_size = 30998800 2024/11/24-02:04:34.388822 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924339 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.20 avg-zval = 31.36 2024/11/24-02:04:34.389623 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-02:04:35.202067 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288187.sst): second pass time = 1.29's, 84.154'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.574'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 279.573'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30961.937'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.154'MB/sec zip pipeline throughput = 84.154'MB/sec entries = 924339 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.20 avg-zval = 31.36 usrkeys = 924339 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.20 avg-zval = 31.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924339 bytes = 3119664 } seq expand size = 6470373 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3759 value = 3.7376 dict = 1.00 all = 3.8377 } Zip/UnZip{ index = 0.0964 value = 0.2676 dict = 1.00 all = 0.2606 } ---------------------------- total value len = 10.569514 GB avg = 0.159 KB (by entry num) total key len = 1.830651 GB avg = 0.028 KB total ukey len = 1.276328 GB avg = 0.016 KB total ukey num = 0.079472933 Billion total entry num = 0.066280662 Billion write speed all = 0.200770168 MB/sec (with seq num) write speed all = 0.192185005 MB/sec (without seq num) 2024/11/24-02:04:35.251800 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288187: 924339 keys, 34973216 bytes, temperature: kWarm 2024/11/24-02:04:35.251872 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385075251838, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288187, "file_size": 34973216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296724763, "largest_seqno": 4385746448, "table_properties": {"data_size": 28985856, "index_size": 1781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881492, "raw_average_key_size": 28, "raw_value_size": 108336266, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924339, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385072, "oldest_key_time": 0, "file_creation_time": 1732385072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288187, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:35.288196 139957542479424 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst) /dev/shm/ToplingTemp/Topling-243955-PEB8MW 2024/11/24-02:04:35.378649 139957542479424 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst): first pass time = 0.09's, 238.553'MB/sec 2024/11/24-02:04:35.586343 139957542479424 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst): old prealloc_size = 147639500, real_size = 5836960 2024/11/24-02:04:35.586488 139957542479424 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 151892 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0173 GB zip-val = 0.0055 GB avg-val = 114.00 avg-zval = 36.26 2024/11/24-02:04:35.586640 139957542479424 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/24-02:04:35.721403 139957542479424 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00/288188.sst): second pass time = 0.21's, 83.655'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 47.671'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2475.210'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.332'MB/sec write SST data time = 0.02's, 311.819'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.181'MB, throughput = 6259.718'MB/sec zip my value time = 0.21's, unzip length = 0.017'GB zip my value throughput = 83.655'MB/sec zip pipeline throughput = 83.655'MB/sec entries = 151892 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 36.26 usrkeys = 151892 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 36.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 151892 bytes = 455696 } seq expand size = 1063244 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0030 GB value = 0.0173 GB dict = 0.18 MB all = 0.0216 GB } __ZipSize{ index = 0.0003 GB value = 0.0055 GB dict = 0.18 MB all = 0.0064 GB } UnZip/Zip{ index = 10.4070 value = 3.1444 dict = 1.00 all = 3.3504 } Zip/UnZip{ index = 0.0961 value = 0.3180 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 10.587893 GB avg = 0.159 KB (by entry num) total key len = 1.834904 GB avg = 0.028 KB total ukey len = 1.279366 GB avg = 0.016 KB total ukey num = 0.079624825 Billion total entry num = 0.066432554 Billion write speed all = 0.201134908 MB/sec (with seq num) write speed all = 0.192530143 MB/sec (without seq num) 2024/11/24-02:04:35.736259 139957542479424 [compaction_job.cc:1948] [default] [JOB 39788] Generated table #288188: 151892 keys, 6437647 bytes, temperature: kWarm 2024/11/24-02:04:35.736313 139957542479424 EVENT_LOG_v1 {"time_micros": 1732385075736285, "cf_name": "default", "job": 39788, "event": "table_file_creation", "file_number": 288188, "file_size": 6437647, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4361684496, "largest_seqno": 4377818029, "table_properties": {"data_size": 5506896, "index_size": 291904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4252976, "raw_average_key_size": 28, "raw_value_size": 17315688, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 151892, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732385075, "oldest_key_time": 0, "file_creation_time": 1732385075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 288188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-02:04:35.747189 139957542479424 [compaction_job.cc:703] job-39788: subcompact[0], size: 0.551341 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39788/att-00 [288173,288174,288175,288176,288177,288178,288179,288180,288181,288182,288183,288184,288185,288186,288187,288188] 2024/11/24-02:04:35.834569 139957542479424 (Original Log Time 2024/11/24-02:03:47.008602) EVENT_LOG_v1 {"time_micros": 1732385027008560, "job": 39788, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [288121], "files_L3": [288163, 288164, 288071, 288072, 288073, 288074, 288075, 288076, 288077, 288078, 288079, 288080, 288081, 288082, 288083, 288165], "score": 0.790354, "input_data_size": 571948683, "oldest_snapshot_seqno": 4386209755} 2024/11/24-02:04:35.834573 139957542479424 (Original Log Time 2024/11/24-02:04:35.833225) [compaction_job.cc:2022] [default] [JOB 39788] Compacted 1@2 + 16@3 files to L3 => 551341083 bytes 2024/11/24-02:04:35.834574 139957542479424 (Original Log Time 2024/11/24-02:04:35.834507) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 11.7 rd, 11.3 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 518.4 +0.0 blob) out(525.8 +0.0 blob), read-write-amplify(39.5) write-amplify(19.4) OK, records in: 14178965, records dropped: 104575 output_compression: Snappy 2024/11/24-02:04:35.834580 139957542479424 (Original Log Time 2024/11/24-02:04:35.834549) EVENT_LOG_v1 {"time_micros": 1732385075834530, "cf": "default", "job": 39788, "event": "compaction_finished", "compaction_time_micros": 48738592, "compaction_time_cpu_micros": 21106918, "output_level": 3, "num_output_files": 16, "total_output_size": 551341083, "num_input_records": 14178965, "num_output_records": 14074390, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 372571940, "file_range_sync_nanos": 0, "file_fsync_nanos": 11693, "file_prepare_write_nanos": 43116, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/24-02:04:35.967172 139957542479424 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732385026,"executesMs":48976,"compactionJobId":39788,"attempt":0,"compactionInputRawBytes":2049685298,"compactionInputZipBytes":571948683,"compactionOutputRawBytes":0,"compactionOutputZipBytes":551341083,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-aedc6dded979480708432887e9d1c632-d8d8350b5af36e4f-00"}}, response = 2024/11/24-02:04:35.967242 139957542479424 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39788/att-00: olev 3, work 48.976 s, result 6.020 ms, install 1.370 ms, input{raw 1.909 GiB zip 545.453 MiB}