2024/11/23-17:24:36.557738 139957592835648 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00: fork to child time = 0.006230 sec 2024/11/23-17:24:36.558389 139957592835648 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00 2024/11/23-17:24:36.565120 139957592835648 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00 2024/11/23-17:24:36.565996 139957592835648 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:24:36.566195 139957592835648 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:24:36.627792 139957592835648 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31873/att-00/MANIFEST-206347 2024/11/23-17:24:36.648324 139957592835648 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31873/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 272142, last_sequence is 4145465524, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 271981 2024/11/23-17:24:36.648329 139957592835648 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:24:36.648484 139957592835648 [version_set.cc:5721] Creating manifest 272142 2024/11/23-17:24:36.685359 139957592835648 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:24:36.708375 139957592835648 [compaction_job.cc:2437] [default] [JOB 31873] Compacting 1@2 + 23@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-17:24:36.708389 139957592835648 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 23855 Base level 2, inputs: [272117(27MB)], [272135(37MB) 272136(4532KB) 272056(37MB) 272057(34MB) 272058(33MB) 272059(34MB) 272060(34MB) 272061(34MB) 272062(38MB) 272063(35MB) 272064(36MB) 272065(33MB) 272066(35MB) 272067(36MB) 272068(35MB) 272069(35MB) 272070(35MB) 272071(35MB) 272072(37MB) 272073(37MB) 272074(37MB) 272075(35MB) 272137(35MB)] 2024/11/23-17:24:36.716429 139957592835648 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:24:36.716453 139957592835648 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:24:36.716467 139957592835648 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:24:36.716472 139957592835648 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:24:36.716479 139957592835648 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:24:37.301680 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst) /dev/shm/ToplingTemp/Topling-132037-DluNAc 2024/11/23-17:24:38.003101 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst): first pass time = 0.70's, 191.364'MB/sec 2024/11/23-17:24:39.539195 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst): old prealloc_size = 147639500, real_size = 35141040 2024/11/23-17:24:39.539780 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926760 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.82 avg-zval = 35.77 2024/11/23-17:24:39.540604 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:40.555059 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272143.sst): second pass time = 1.54's, 70.523'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 38.630'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3108.157'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.13's, 310.116'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33081.116'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.523'MB/sec zip pipeline throughput = 70.523'MB/sec entries = 926760 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 35.77 usrkeys = 926760 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 35.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926760 bytes = 3243680 } seq expand size = 6487320 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5164 value = 3.2663 dict = 1.00 all = 3.4202 } Zip/UnZip{ index = 0.0951 value = 0.3062 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 8.963376 GB avg = 0.168 KB (by entry num) total key len = 1.466770 GB avg = 0.028 KB total ukey len = 1.016413 GB avg = 0.015 KB total ukey num = 0.066477195 Billion total entry num = 0.053284924 Billion write speed all = 0.341207468 MB/sec (with seq num) write speed all = 0.327262341 MB/sec (without seq num) 2024/11/23-17:24:40.604102 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272143: 926760 keys, 39242778 bytes, temperature: kWarm 2024/11/23-17:24:40.604197 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353880604138, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272143, "file_size": 39242778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3967653232, "largest_seqno": 4145240855, "table_properties": {"data_size": 33146656, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949280, "raw_average_key_size": 28, "raw_value_size": 108268552, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926760, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353877, "oldest_key_time": 0, "file_creation_time": 1732353877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272143, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:40.642193 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst) /dev/shm/ToplingTemp/Topling-132037-6QaG3c 2024/11/23-17:24:41.734885 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst): first pass time = 1.09's, 122.837'MB/sec 2024/11/23-17:24:43.241711 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst): old prealloc_size = 147639500, real_size = 36527600 2024/11/23-17:24:43.242442 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934034 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 115.70 avg-zval = 36.93 2024/11/23-17:24:43.243173 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:44.188574 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272144.sst): second pass time = 1.51's, 71.767'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2524.933'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.297'MB/sec write SST data time = 0.16's, 251.558'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34114.457'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.767'MB/sec zip pipeline throughput = 71.767'MB/sec entries = 934034 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.70 avg-zval = 36.93 usrkeys = 934034 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.88 avg-zval = 36.93 TagRS{ kinds = RS_Key0_TagN bytes = 146040 } TagArray{ size = 692072 bytes = 2508768 } seq expand size = 4844504 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3774 value = 3.1329 dict = 1.00 all = 3.3524 } Zip/UnZip{ index = 0.0964 value = 0.3192 dict = 1.00 all = 0.2983 } ---------------------------- total value len = 9.076285 GB avg = 0.167 KB (by entry num) total key len = 1.492923 GB avg = 0.028 KB total ukey len = 1.035094 GB avg = 0.015 KB total ukey num = 0.067411229 Billion total entry num = 0.054218958 Billion write speed all = 0.345715603 MB/sec (with seq num) write speed all = 0.331527717 MB/sec (without seq num) 2024/11/23-17:24:44.240311 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272144: 934034 keys, 40036838 bytes, temperature: kWarm 2024/11/23-17:24:44.240380 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353884240344, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272144, "file_size": 40036838, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34493792, "index_size": 1800128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26152952, "raw_average_key_size": 28, "raw_value_size": 108064908, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934034, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353880, "oldest_key_time": 0, "file_creation_time": 1732353880, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272144, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:44.278120 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst) /dev/shm/ToplingTemp/Topling-132037-mIJU7d 2024/11/23-17:24:45.522864 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst): first pass time = 1.24's, 107.831'MB/sec 2024/11/23-17:24:47.068730 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst): old prealloc_size = 147639500, real_size = 34953808 2024/11/23-17:24:47.069506 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081611 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.09 avg-zval = 30.16 2024/11/23-17:24:47.069562 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:47.907899 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272145.sst): second pass time = 1.54's, 67.299'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.579'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2721.723'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.403'MB/sec write SST data time = 0.17's, 213.530'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.035'MB, throughput = 33305.275'MB/sec zip my value time = 1.54's, unzip length = 0.104'GB zip my value throughput = 67.299'MB/sec zip pipeline throughput = 67.299'MB/sec entries = 1081611 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.16 usrkeys = 1081611 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.04 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5163 value = 3.1856 dict = 1.00 all = 3.7565 } Zip/UnZip{ index = 0.0951 value = 0.3139 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 9.180218 GB avg = 0.166 KB (by entry num) total key len = 1.523208 GB avg = 0.028 KB total ukey len = 1.056726 GB avg = 0.015 KB total ukey num = 0.068492840 Billion total entry num = 0.055300569 Billion write speed all = 0.350063239 MB/sec (with seq num) write speed all = 0.335594081 MB/sec (without seq num) 2024/11/23-17:24:47.954339 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272145: 1081611 keys, 35729735 bytes, temperature: kWarm 2024/11/23-17:24:47.954450 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353887954407, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272145, "file_size": 35729735, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32626176, "index_size": 2057024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30285108, "raw_average_key_size": 28, "raw_value_size": 103932731, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353884, "oldest_key_time": 0, "file_creation_time": 1732353884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272145, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:47.993871 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst) /dev/shm/ToplingTemp/Topling-132037-hWwujf 2024/11/23-17:24:49.201569 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst): first pass time = 1.21's, 111.139'MB/sec 2024/11/23-17:24:50.523361 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst): old prealloc_size = 147639500, real_size = 31787328 2024/11/23-17:24:50.524058 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925652 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.00 avg-zval = 32.19 2024/11/23-17:24:50.524940 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:51.622263 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272146.sst): second pass time = 1.32's, 82.050'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 32.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2607.889'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.247'MB/sec write SST data time = 0.16's, 225.053'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37470.301'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.050'MB/sec zip pipeline throughput = 82.050'MB/sec entries = 925652 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 32.19 usrkeys = 925652 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.51 avg-zval = 32.19 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 860822 bytes = 3228096 } seq expand size = 6025754 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5134 value = 3.6348 dict = 1.00 all = 3.7263 } Zip/UnZip{ index = 0.0951 value = 0.2751 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 9.294543 GB avg = 0.165 KB (by entry num) total key len = 1.549127 GB avg = 0.028 KB total ukey len = 1.075239 GB avg = 0.015 KB total ukey num = 0.069418492 Billion total entry num = 0.056226221 Billion write speed all = 0.354606925 MB/sec (with seq num) write speed all = 0.339897361 MB/sec (without seq num) 2024/11/23-17:24:51.667894 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272146: 925652 keys, 36019118 bytes, temperature: kWarm 2024/11/23-17:24:51.667964 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353891667929, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272146, "file_size": 36019118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 29794864, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918256, "raw_average_key_size": 28, "raw_value_size": 108299480, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925652, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353887, "oldest_key_time": 0, "file_creation_time": 1732353887, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272146, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:51.703817 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst) /dev/shm/ToplingTemp/Topling-132037-G0mm8b 2024/11/23-17:24:52.912963 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst): first pass time = 1.21's, 111.006'MB/sec 2024/11/23-17:24:54.362556 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst): old prealloc_size = 147639500, real_size = 32281488 2024/11/23-17:24:54.363273 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.22 avg-zval = 32.53 2024/11/23-17:24:54.364158 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:55.289484 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272147.sst): second pass time = 1.45's, 74.668'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2572.316'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.15's, 239.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26886.594'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.668'MB/sec zip pipeline throughput = 74.668'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 32.53 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5136 value = 3.5722 dict = 1.00 all = 3.6766 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 9.409217 GB avg = 0.165 KB (by entry num) total key len = 1.575185 GB avg = 0.028 KB total ukey len = 1.093852 GB avg = 0.016 KB total ukey num = 0.070349138 Billion total entry num = 0.057156867 Billion write speed all = 0.359166044 MB/sec (with seq num) write speed all = 0.344214802 MB/sec (without seq num) 2024/11/23-17:24:55.335628 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272147: 930646 keys, 36506258 bytes, temperature: kWarm 2024/11/23-17:24:55.335694 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353895335660, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272147, "file_size": 36506258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30278272, "index_size": 1770368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353891, "oldest_key_time": 0, "file_creation_time": 1732353891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272147, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:55.372754 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst) /dev/shm/ToplingTemp/Topling-132037-OBXZFf 2024/11/23-17:24:56.473066 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst): first pass time = 1.10's, 121.986'MB/sec 2024/11/23-17:24:57.983353 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst): old prealloc_size = 147639500, real_size = 32401376 2024/11/23-17:24:57.984085 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.97 avg-zval = 32.60 2024/11/23-17:24:57.984943 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:24:58.890408 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272148.sst): second pass time = 1.51's, 71.664'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.257'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2496.947'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.15's, 237.334'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34428.836'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.664'MB/sec zip pipeline throughput = 71.664'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 32.60 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5118 value = 3.5571 dict = 1.00 all = 3.6757 } Zip/UnZip{ index = 0.0951 value = 0.2811 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 9.523858 GB avg = 0.164 KB (by entry num) total key len = 1.601287 GB avg = 0.028 KB total ukey len = 1.112497 GB avg = 0.016 KB total ukey num = 0.071281378 Billion total entry num = 0.058089107 Billion write speed all = 0.363725223 MB/sec (with seq num) write speed all = 0.348531912 MB/sec (without seq num) 2024/11/23-17:24:58.936299 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272148: 932240 keys, 36514906 bytes, temperature: kWarm 2024/11/23-17:24:58.936364 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353898936332, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272148, "file_size": 36514906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30394448, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353895, "oldest_key_time": 0, "file_creation_time": 1732353895, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272148, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:24:58.974214 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst) /dev/shm/ToplingTemp/Topling-132037-BzeK8b 2024/11/23-17:25:00.141159 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst): first pass time = 1.17's, 115.020'MB/sec 2024/11/23-17:25:01.883709 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst): old prealloc_size = 147639500, real_size = 33097760 2024/11/23-17:25:01.884309 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925501 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.02 avg-zval = 33.61 2024/11/23-17:25:01.885151 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:02.739209 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272149.sst): second pass time = 1.74's, 62.181'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.745'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3062.488'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34871.151'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.181'MB/sec zip pipeline throughput = 103.899'MB/sec entries = 925501 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 33.61 usrkeys = 925501 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925501 bytes = 3470640 } seq expand size = 6478507 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5129 value = 3.4818 dict = 1.00 all = 3.5865 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 9.638640 GB avg = 0.163 KB (by entry num) total key len = 1.627202 GB avg = 0.028 KB total ukey len = 1.131007 GB avg = 0.016 KB total ukey num = 0.072206879 Billion total entry num = 0.059014608 Billion write speed all = 0.368278803 MB/sec (with seq num) write speed all = 0.352845369 MB/sec (without seq num) 2024/11/23-17:25:02.785841 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272149: 925501 keys, 37423178 bytes, temperature: kWarm 2024/11/23-17:25:02.785915 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353902785878, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272149, "file_size": 37423178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31105488, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914028, "raw_average_key_size": 28, "raw_value_size": 108303794, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925501, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353898, "oldest_key_time": 0, "file_creation_time": 1732353898, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272149, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:02.838996 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst) /dev/shm/ToplingTemp/Topling-132037-R0fTkc 2024/11/23-17:25:04.020799 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst): first pass time = 1.18's, 113.577'MB/sec 2024/11/23-17:25:05.680165 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst): old prealloc_size = 147639500, real_size = 35253936 2024/11/23-17:25:05.680747 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929432 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.41 avg-zval = 35.77 2024/11/23-17:25:05.682801 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:06.714186 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272150.sst): second pass time = 1.66's, 65.231'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.177'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3264.829'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.13's, 295.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26763.810'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.231'MB/sec zip pipeline throughput = 95.738'MB/sec entries = 929432 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.41 avg-zval = 35.77 usrkeys = 929432 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.41 avg-zval = 35.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929432 bytes = 3369200 } seq expand size = 6506024 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4444 value = 3.2548 dict = 1.00 all = 3.4001 } Zip/UnZip{ index = 0.0957 value = 0.3072 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 9.753340 GB avg = 0.163 KB (by entry num) total key len = 1.653226 GB avg = 0.028 KB total ukey len = 1.149596 GB avg = 0.016 KB total ukey num = 0.073136311 Billion total entry num = 0.059944040 Billion write speed all = 0.372830600 MB/sec (with seq num) write speed all = 0.357156138 MB/sec (without seq num) 2024/11/23-17:25:06.763270 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272150: 929432 keys, 39475216 bytes, temperature: kWarm 2024/11/23-17:25:06.763341 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353906763304, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272150, "file_size": 39475216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33241632, "index_size": 1779776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024096, "raw_average_key_size": 28, "raw_value_size": 108193716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353902, "oldest_key_time": 0, "file_creation_time": 1732353902, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272150, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:06.808025 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst) /dev/shm/ToplingTemp/Topling-132037-mmykhe 2024/11/23-17:25:07.704087 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst): first pass time = 0.90's, 149.795'MB/sec 2024/11/23-17:25:09.281364 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst): old prealloc_size = 147639500, real_size = 32681776 2024/11/23-17:25:09.281899 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.55 avg-zval = 33.05 2024/11/23-17:25:09.284053 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:10.296689 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272151.sst): second pass time = 1.58's, 68.643'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.459'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3463.275'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.262'MB/sec write SST data time = 0.13's, 284.739'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33242.072'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.643'MB/sec zip pipeline throughput = 68.643'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.05 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5150 value = 3.5270 dict = 1.00 all = 3.6258 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 9.868059 GB avg = 0.162 KB (by entry num) total key len = 1.679224 GB avg = 0.028 KB total ukey len = 1.168166 GB avg = 0.016 KB total ukey num = 0.074064831 Billion total entry num = 0.060872560 Billion write speed all = 0.377385848 MB/sec (with seq num) write speed all = 0.361470455 MB/sec (without seq num) 2024/11/23-17:25:10.342599 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272151: 928520 keys, 37017060 bytes, temperature: kWarm 2024/11/23-17:25:10.342670 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353910342634, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272151, "file_size": 37017060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30683360, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353906, "oldest_key_time": 0, "file_creation_time": 1732353906, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272151, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:10.389583 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst) /dev/shm/ToplingTemp/Topling-132037-oxYQre 2024/11/23-17:25:11.488252 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst): first pass time = 1.10's, 122.169'MB/sec 2024/11/23-17:25:12.990269 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst): old prealloc_size = 147639500, real_size = 35091040 2024/11/23-17:25:12.990961 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925703 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 116.99 avg-zval = 35.74 2024/11/23-17:25:12.991771 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:13.937029 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272152.sst): second pass time = 1.50's, 72.148'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.067'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2580.999'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.17's, 234.944'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36919.224'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.148'MB/sec zip pipeline throughput = 72.148'MB/sec entries = 925703 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.99 avg-zval = 35.74 usrkeys = 925703 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.61 avg-zval = 35.74 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 742706 bytes = 2785168 } seq expand size = 5198942 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4438 value = 3.2732 dict = 1.00 all = 3.4521 } Zip/UnZip{ index = 0.0958 value = 0.3055 dict = 1.00 all = 0.2897 } ---------------------------- total value len = 9.981556 GB avg = 0.162 KB (by entry num) total key len = 1.705144 GB avg = 0.028 KB total ukey len = 1.186680 GB avg = 0.016 KB total ukey num = 0.074990534 Billion total entry num = 0.061798263 Billion write speed all = 0.381896803 MB/sec (with seq num) write speed all = 0.365741303 MB/sec (without seq num) 2024/11/23-17:25:13.986132 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272152: 925703 keys, 38879598 bytes, temperature: kWarm 2024/11/23-17:25:13.986198 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353913986165, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272152, "file_size": 38879598, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33086672, "index_size": 1772736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919684, "raw_average_key_size": 28, "raw_value_size": 108298174, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925703, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353910, "oldest_key_time": 0, "file_creation_time": 1732353910, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272152, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:14.022691 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst) /dev/shm/ToplingTemp/Topling-132037-toS0Tb 2024/11/23-17:25:15.173144 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst): first pass time = 1.15's, 116.669'MB/sec 2024/11/23-17:25:16.532750 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst): old prealloc_size = 147639500, real_size = 31988304 2024/11/23-17:25:16.533347 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 115.79 avg-zval = 32.12 2024/11/23-17:25:16.535511 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:17.386692 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272153.sst): second pass time = 1.36's, 79.585'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.419'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3093.656'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.322'MB/sec write SST data time = 0.13's, 282.239'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38614.249'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.585'MB/sec zip pipeline throughput = 79.585'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.12 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5142 value = 3.6052 dict = 1.00 all = 3.7054 } Zip/UnZip{ index = 0.0951 value = 0.2774 dict = 1.00 all = 0.2699 } ---------------------------- total value len = 10.096171 GB avg = 0.161 KB (by entry num) total key len = 1.731281 GB avg = 0.028 KB total ukey len = 1.205349 GB avg = 0.016 KB total ukey num = 0.075923991 Billion total entry num = 0.062731720 Billion write speed all = 0.386452720 MB/sec (with seq num) write speed all = 0.370055041 MB/sec (without seq num) 2024/11/23-17:25:17.430915 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272153: 933457 keys, 36222586 bytes, temperature: kWarm 2024/11/23-17:25:17.431005 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353917430949, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272153, "file_size": 36222586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29979136, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353914, "oldest_key_time": 0, "file_creation_time": 1732353914, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272153, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:17.481136 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst) /dev/shm/ToplingTemp/Topling-132037-ziz2Rc 2024/11/23-17:25:18.382418 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst): first pass time = 0.90's, 148.926'MB/sec 2024/11/23-17:25:19.695530 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst): old prealloc_size = 147639500, real_size = 33220752 2024/11/23-17:25:19.696130 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932359 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.96 avg-zval = 33.47 2024/11/23-17:25:19.698241 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:20.545230 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272154.sst): second pass time = 1.31's, 82.385'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.077'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.860'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 293.044'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34708.986'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.385'MB/sec zip pipeline throughput = 82.385'MB/sec entries = 932359 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.96 avg-zval = 33.47 usrkeys = 932359 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.96 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932359 bytes = 3379824 } seq expand size = 6526513 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4461 value = 3.4649 dict = 1.00 all = 3.5838 } Zip/UnZip{ index = 0.0957 value = 0.2886 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 10.210810 GB avg = 0.160 KB (by entry num) total key len = 1.757387 GB avg = 0.028 KB total ukey len = 1.223996 GB avg = 0.016 KB total ukey num = 0.076856350 Billion total entry num = 0.063664079 Billion write speed all = 0.391011077 MB/sec (with seq num) write speed all = 0.374371404 MB/sec (without seq num) 2024/11/23-17:25:20.591135 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272154: 932359 keys, 37450914 bytes, temperature: kWarm 2024/11/23-17:25:20.591204 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353920591168, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272154, "file_size": 37450914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 31202368, "index_size": 1785088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26106052, "raw_average_key_size": 28, "raw_value_size": 108111782, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932359, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353917, "oldest_key_time": 0, "file_creation_time": 1732353917, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272154, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:20.629204 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst) /dev/shm/ToplingTemp/Topling-132037-GrUYrb 2024/11/23-17:25:21.659232 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst): first pass time = 1.03's, 130.310'MB/sec 2024/11/23-17:25:22.993739 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst): old prealloc_size = 147639500, real_size = 34811040 2024/11/23-17:25:22.994335 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 117.01 avg-zval = 35.43 2024/11/23-17:25:22.996449 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:23.841011 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272155.sst): second pass time = 1.33's, 81.241'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.194'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3119.918'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 307.342'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33398.677'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.241'MB/sec zip pipeline throughput = 81.241'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.01 avg-zval = 35.43 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.01 avg-zval = 35.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.3741 value = 3.3024 dict = 1.00 all = 3.4294 } Zip/UnZip{ index = 0.0964 value = 0.3028 dict = 1.00 all = 0.2916 } ---------------------------- total value len = 10.325590 GB avg = 0.160 KB (by entry num) total key len = 1.783303 GB avg = 0.028 KB total ukey len = 1.242508 GB avg = 0.016 KB total ukey num = 0.077781918 Billion total entry num = 0.064589647 Billion write speed all = 0.395565165 MB/sec (with seq num) write speed all = 0.378685396 MB/sec (without seq num) 2024/11/23-17:25:23.889378 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272155: 925568 keys, 39136850 bytes, temperature: kWarm 2024/11/23-17:25:23.889457 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353923889415, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272155, "file_size": 39136850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32795088, "index_size": 1784384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353920, "oldest_key_time": 0, "file_creation_time": 1732353920, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272155, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:23.928218 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst) /dev/shm/ToplingTemp/Topling-132037-IHAafd 2024/11/23-17:25:24.918219 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst): first pass time = 0.99's, 135.578'MB/sec 2024/11/23-17:25:26.240329 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst): old prealloc_size = 147639500, real_size = 32655312 2024/11/23-17:25:26.240915 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931676 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.06 avg-zval = 32.90 2024/11/23-17:25:26.243048 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:27.099763 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272156.sst): second pass time = 1.32's, 81.834'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.918'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.862'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 288.176'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35009.059'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.834'MB/sec zip pipeline throughput = 81.834'MB/sec entries = 931676 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.90 usrkeys = 931676 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931676 bytes = 3377344 } seq expand size = 6521732 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5127 value = 3.5280 dict = 1.00 all = 3.6389 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 10.440243 GB avg = 0.159 KB (by entry num) total key len = 1.809390 GB avg = 0.028 KB total ukey len = 1.261141 GB avg = 0.016 KB total ukey num = 0.078713594 Billion total entry num = 0.065521323 Billion write speed all = 0.400120153 MB/sec (with seq num) write speed all = 0.382998724 MB/sec (without seq num) 2024/11/23-17:25:27.147437 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272156: 931676 keys, 36883828 bytes, temperature: kWarm 2024/11/23-17:25:27.147508 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353927147472, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272156, "file_size": 36883828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30649728, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086928, "raw_average_key_size": 28, "raw_value_size": 108130844, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931676, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353923, "oldest_key_time": 0, "file_creation_time": 1732353923, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272156, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:27.189733 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst) /dev/shm/ToplingTemp/Topling-132037-H8ZxBb 2024/11/23-17:25:28.215597 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst): first pass time = 1.03's, 130.839'MB/sec 2024/11/23-17:25:29.576246 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst): old prealloc_size = 147639500, real_size = 34532288 2024/11/23-17:25:29.576801 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923008 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.41 avg-zval = 35.25 2024/11/23-17:25:29.578987 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:30.423038 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272157.sst): second pass time = 1.36's, 79.717'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.883'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3336.788'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 303.597'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 37202.364'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.717'MB/sec zip pipeline throughput = 79.717'MB/sec entries = 923008 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.41 avg-zval = 35.25 usrkeys = 923008 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.41 avg-zval = 35.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923008 bytes = 3461296 } seq expand size = 6461056 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4450 value = 3.3311 dict = 1.00 all = 3.4546 } Zip/UnZip{ index = 0.0957 value = 0.3002 dict = 1.00 all = 0.2895 } ---------------------------- total value len = 10.555078 GB avg = 0.159 KB (by entry num) total key len = 1.835234 GB avg = 0.028 KB total ukey len = 1.279601 GB avg = 0.016 KB total ukey num = 0.079636602 Billion total entry num = 0.066444331 Billion write speed all = 0.404671340 MB/sec (with seq num) write speed all = 0.387310603 MB/sec (without seq num) 2024/11/23-17:25:30.472989 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272157: 923008 keys, 38851532 bytes, temperature: kWarm 2024/11/23-17:25:30.473059 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353930473024, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272157, "file_size": 38851532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 32534000, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844224, "raw_average_key_size": 28, "raw_value_size": 108373600, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923008, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353927, "oldest_key_time": 0, "file_creation_time": 1732353927, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272157, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:30.510610 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst) /dev/shm/ToplingTemp/Topling-132037-hrlW7b 2024/11/23-17:25:31.344087 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst): first pass time = 0.83's, 161.040'MB/sec 2024/11/23-17:25:32.636063 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst): old prealloc_size = 147639500, real_size = 32932416 2024/11/23-17:25:32.636648 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930338 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.27 avg-zval = 33.23 2024/11/23-17:25:32.637657 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:33.496788 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272158.sst): second pass time = 1.29's, 83.775'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.305'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.486'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 294.860'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34081.924'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.775'MB/sec zip pipeline throughput = 83.775'MB/sec entries = 930338 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.27 avg-zval = 33.23 usrkeys = 930338 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.27 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930338 bytes = 3488784 } seq expand size = 6512366 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4433 value = 3.4986 dict = 1.00 all = 3.6010 } Zip/UnZip{ index = 0.0958 value = 0.2858 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 10.669758 GB avg = 0.158 KB (by entry num) total key len = 1.861283 GB avg = 0.028 KB total ukey len = 1.298208 GB avg = 0.016 KB total ukey num = 0.080566940 Billion total entry num = 0.067374669 Billion write speed all = 0.409226547 MB/sec (with seq num) write speed all = 0.391624497 MB/sec (without seq num) 2024/11/23-17:25:33.542291 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272158: 930338 keys, 37272702 bytes, temperature: kWarm 2024/11/23-17:25:33.542362 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353933542326, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272158, "file_size": 37272702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30917936, "index_size": 1781696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26049464, "raw_average_key_size": 28, "raw_value_size": 108168332, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353930, "oldest_key_time": 0, "file_creation_time": 1732353930, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272158, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:33.581705 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst) /dev/shm/ToplingTemp/Topling-132037-c5lZXe 2024/11/23-17:25:34.600004 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst): first pass time = 1.02's, 131.810'MB/sec 2024/11/23-17:25:36.002898 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst): old prealloc_size = 147639500, real_size = 34916016 2024/11/23-17:25:36.003509 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928795 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0329 GB avg-val = 116.51 avg-zval = 35.43 2024/11/23-17:25:36.003997 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:36.781337 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272159.sst): second pass time = 1.40's, 77.177'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 48.231'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3018.580'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.356'MB/sec write SST data time = 0.13's, 297.943'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30262.493'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.177'MB/sec zip pipeline throughput = 77.177'MB/sec entries = 928795 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.51 avg-zval = 35.43 usrkeys = 928795 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.00 avg-zval = 35.43 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 464016 bytes = 1624064 } seq expand size = 3248112 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4428 value = 3.2886 dict = 1.00 all = 3.5750 } Zip/UnZip{ index = 0.0958 value = 0.3041 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 10.781218 GB avg = 0.158 KB (by entry num) total key len = 1.887289 GB avg = 0.028 KB total ukey len = 1.316784 GB avg = 0.016 KB total ukey num = 0.081495735 Billion total entry num = 0.068303464 Billion write speed all = 0.413671402 MB/sec (with seq num) write speed all = 0.395828612 MB/sec (without seq num) 2024/11/23-17:25:36.827005 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272159: 928795 keys, 37543363 bytes, temperature: kWarm 2024/11/23-17:25:36.827074 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353936827039, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272159, "file_size": 37543363, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32904800, "index_size": 1778816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006260, "raw_average_key_size": 28, "raw_value_size": 108211486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928795, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353933, "oldest_key_time": 0, "file_creation_time": 1732353933, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272159, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:36.878409 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst) /dev/shm/ToplingTemp/Topling-132037-bCn3Gf 2024/11/23-17:25:37.892170 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst): first pass time = 1.01's, 132.403'MB/sec 2024/11/23-17:25:39.401045 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst): old prealloc_size = 147639500, real_size = 36867120 2024/11/23-17:25:39.401619 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926822 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.82 avg-zval = 37.63 2024/11/23-17:25:39.401848 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:40.117997 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272160.sst): second pass time = 1.51's, 71.810'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.746'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.391'MB/sec write SST data time = 0.12's, 311.725'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35789.052'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.810'MB/sec zip pipeline throughput = 71.810'MB/sec entries = 926822 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 37.63 usrkeys = 926822 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.98 avg-zval = 37.63 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5152 value = 3.1047 dict = 1.00 all = 3.4996 } Zip/UnZip{ index = 0.0951 value = 0.3221 dict = 1.00 all = 0.2857 } ---------------------------- total value len = 10.890565 GB avg = 0.157 KB (by entry num) total key len = 1.913240 GB avg = 0.028 KB total ukey len = 1.335320 GB avg = 0.016 KB total ukey num = 0.082422557 Billion total entry num = 0.069230286 Billion write speed all = 0.418043828 MB/sec (with seq num) write speed all = 0.399960896 MB/sec (without seq num) 2024/11/23-17:25:40.164964 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272160: 926822 keys, 38352441 bytes, temperature: kWarm 2024/11/23-17:25:40.165042 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353940165006, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272160, "file_size": 38352441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34872416, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951016, "raw_average_key_size": 28, "raw_value_size": 108266732, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353936, "oldest_key_time": 0, "file_creation_time": 1732353936, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272160, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:40.203096 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst) /dev/shm/ToplingTemp/Topling-132037-IBhcdc 2024/11/23-17:25:41.224497 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst): first pass time = 1.02's, 131.410'MB/sec 2024/11/23-17:25:42.569598 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst): old prealloc_size = 147639500, real_size = 35849552 2024/11/23-17:25:42.570143 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926783 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 116.82 avg-zval = 36.53 2024/11/23-17:25:42.570947 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:43.393481 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272161.sst): second pass time = 1.34's, 80.537'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.265'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.080'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 309.191'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37379.957'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.537'MB/sec zip pipeline throughput = 80.537'MB/sec entries = 926783 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 36.53 usrkeys = 926783 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.33 avg-zval = 36.53 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 862393 bytes = 2910592 } seq expand size = 6036751 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.5129 value = 3.1980 dict = 1.00 all = 3.3754 } Zip/UnZip{ index = 0.0951 value = 0.3127 dict = 1.00 all = 0.2963 } ---------------------------- total value len = 11.004870 GB avg = 0.157 KB (by entry num) total key len = 1.939190 GB avg = 0.028 KB total ukey len = 1.353856 GB avg = 0.016 KB total ukey num = 0.083349340 Billion total entry num = 0.070157069 Billion write speed all = 0.422577944 MB/sec (with seq num) write speed all = 0.404254896 MB/sec (without seq num) 2024/11/23-17:25:43.440987 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272161: 926783 keys, 39763102 bytes, temperature: kWarm 2024/11/23-17:25:43.441057 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353943441023, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272161, "file_size": 39763102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4145240937, "table_properties": {"data_size": 33854528, "index_size": 1763136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949924, "raw_average_key_size": 28, "raw_value_size": 108267938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353940, "oldest_key_time": 0, "file_creation_time": 1732353940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272161, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:43.481622 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst) /dev/shm/ToplingTemp/Topling-132037-wnJ2Fe 2024/11/23-17:25:44.530632 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst): first pass time = 1.05's, 127.951'MB/sec 2024/11/23-17:25:45.880067 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst): old prealloc_size = 147639500, real_size = 35566144 2024/11/23-17:25:45.880631 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928617 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0336 GB avg-val = 116.54 avg-zval = 36.15 2024/11/23-17:25:45.882470 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:46.716323 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272162.sst): second pass time = 1.35's, 80.266'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.161'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.255'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 306.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35316.151'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.266'MB/sec zip pipeline throughput = 80.266'MB/sec entries = 928617 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 36.15 usrkeys = 928617 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 36.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928617 bytes = 3018016 } seq expand size = 6500319 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.5146 value = 3.2238 dict = 1.00 all = 3.4033 } Zip/UnZip{ index = 0.0951 value = 0.3102 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 11.119587 GB avg = 0.156 KB (by entry num) total key len = 1.965192 GB avg = 0.028 KB total ukey len = 1.372428 GB avg = 0.016 KB total ukey num = 0.084277957 Billion total entry num = 0.071085686 Billion write speed all = 0.427125558 MB/sec (with seq num) write speed all = 0.408561996 MB/sec (without seq num) 2024/11/23-17:25:46.764774 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272162: 928617 keys, 39437596 bytes, temperature: kWarm 2024/11/23-17:25:46.764841 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353946764805, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272162, "file_size": 39437596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4145240982, "table_properties": {"data_size": 33567472, "index_size": 1766336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26001276, "raw_average_key_size": 28, "raw_value_size": 108216458, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928617, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353943, "oldest_key_time": 0, "file_creation_time": 1732353943, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272162, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:46.800660 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst) /dev/shm/ToplingTemp/Topling-132037-fEgPMb 2024/11/23-17:25:47.693927 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst): first pass time = 0.89's, 150.262'MB/sec 2024/11/23-17:25:49.017380 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst): old prealloc_size = 147639500, real_size = 34417888 2024/11/23-17:25:49.017927 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924382 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.20 avg-zval = 35.08 2024/11/23-17:25:49.019946 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:49.855533 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272163.sst): second pass time = 1.32's, 81.958'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.814'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3347.392'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 303.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35222.009'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.958'MB/sec zip pipeline throughput = 81.958'MB/sec entries = 924382 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 35.08 usrkeys = 924382 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 35.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924382 bytes = 3119808 } seq expand size = 6470674 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5124 value = 3.3408 dict = 1.00 all = 3.4954 } Zip/UnZip{ index = 0.0951 value = 0.2993 dict = 1.00 all = 0.2861 } ---------------------------- total value len = 11.234393 GB avg = 0.156 KB (by entry num) total key len = 1.991074 GB avg = 0.028 KB total ukey len = 1.390916 GB avg = 0.016 KB total ukey num = 0.085202339 Billion total entry num = 0.072010068 Billion write speed all = 0.431673810 MB/sec (with seq num) write speed all = 0.412870778 MB/sec (without seq num) 2024/11/23-17:25:49.904673 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272163: 924382 keys, 38398212 bytes, temperature: kWarm 2024/11/23-17:25:49.904744 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353949904709, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272163, "file_size": 38398212, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4145241016, "table_properties": {"data_size": 32427984, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882696, "raw_average_key_size": 28, "raw_value_size": 108335172, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924382, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353946, "oldest_key_time": 0, "file_creation_time": 1732353946, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272163, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:49.941270 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst) /dev/shm/ToplingTemp/Topling-132037-hPxTKb 2024/11/23-17:25:50.939367 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst): first pass time = 1.00's, 134.479'MB/sec 2024/11/23-17:25:52.228494 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst): old prealloc_size = 147639500, real_size = 32472672 2024/11/23-17:25:52.229083 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933395 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.80 avg-zval = 32.64 2024/11/23-17:25:52.231025 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:25:53.087652 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272164.sst): second pass time = 1.29's, 83.891'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3141.448'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 288.499'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34989.288'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.891'MB/sec zip pipeline throughput = 83.891'MB/sec entries = 933395 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 32.64 usrkeys = 933395 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.80 avg-zval = 32.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933395 bytes = 3150224 } seq expand size = 6533765 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5146 value = 3.5479 dict = 1.00 all = 3.6799 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 11.349009 GB avg = 0.156 KB (by entry num) total key len = 2.017209 GB avg = 0.028 KB total ukey len = 1.409584 GB avg = 0.016 KB total ukey num = 0.086135734 Billion total entry num = 0.072943463 Billion write speed all = 0.436221864 MB/sec (with seq num) write speed all = 0.417177116 MB/sec (without seq num) 2024/11/23-17:25:53.133457 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272164: 933395 keys, 36473288 bytes, temperature: kWarm 2024/11/23-17:25:53.133526 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353953133490, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272164, "file_size": 36473288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4145240751, "table_properties": {"data_size": 30463696, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26135060, "raw_average_key_size": 28, "raw_value_size": 108082746, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933395, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353949, "oldest_key_time": 0, "file_creation_time": 1732353949, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272164, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:53.173538 139957592835648 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst) /dev/shm/ToplingTemp/Topling-132037-yko57c 2024/11/23-17:25:53.324173 139957592835648 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst): first pass time = 0.15's, 243.780'MB/sec 2024/11/23-17:25:53.677461 139957592835648 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst): old prealloc_size = 147639500, real_size = 10376816 2024/11/23-17:25:53.677665 139957592835648 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 254172 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0296 GB zip-val = 0.0098 GB avg-val = 116.44 avg-zval = 38.66 2024/11/23-17:25:53.678637 139957592835648 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.037 GB 2024/11/23-17:25:53.903741 139957592835648 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00/272165.sst): second pass time = 0.35's, 83.940'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 50.437'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2702.907'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.336'MB/sec write SST data time = 0.04's, 309.605'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.302'MB, throughput = 9589.693'MB/sec zip my value time = 0.35's, unzip length = 0.030'GB zip my value throughput = 83.940'MB/sec zip pipeline throughput = 83.940'MB/sec entries = 254172 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.44 avg-zval = 38.66 usrkeys = 254172 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.44 avg-zval = 38.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 254172 bytes = 794304 } seq expand size = 1779204 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0051 GB value = 0.0296 GB dict = 0.30 MB all = 0.0367 GB } __ZipSize{ index = 0.0005 GB value = 0.0098 GB dict = 0.30 MB all = 0.0114 GB } UnZip/Zip{ index = 10.4196 value = 3.0121 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.0960 value = 0.3320 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 11.380383 GB avg = 0.155 KB (by entry num) total key len = 2.024326 GB avg = 0.028 KB total ukey len = 1.414667 GB avg = 0.016 KB total ukey num = 0.086389906 Billion total entry num = 0.073197635 Billion write speed all = 0.437466403 MB/sec (with seq num) write speed all = 0.418355803 MB/sec (without seq num) 2024/11/23-17:25:53.923295 139957592835648 [compaction_job.cc:1948] [default] [JOB 31873] Generated table #272165: 254172 keys, 11412359 bytes, temperature: kWarm 2024/11/23-17:25:53.923355 139957592835648 EVENT_LOG_v1 {"time_micros": 1732353953923324, "cf_name": "default", "job": 31873, "event": "table_file_creation", "file_number": 272165, "file_size": 11412359, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223986, "largest_seqno": 4145240773, "table_properties": {"data_size": 9825248, "index_size": 487872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7116816, "raw_average_key_size": 28, "raw_value_size": 29594756, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 254172, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732353953, "oldest_key_time": 0, "file_creation_time": 1732353953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 272165, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:25:53.937830 139957592835648 [compaction_job.cc:703] job-31873: subcompact[0], size: 0.844043 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-31873/att-00 [272143,272144,272145,272146,272147,272148,272149,272150,272151,272152,272153,272154,272155,272156,272157,272158,272159,272160,272161,272162,272163,272164,272165] 2024/11/23-17:25:54.074418 139957592835648 (Original Log Time 2024/11/23-17:24:36.708431) EVENT_LOG_v1 {"time_micros": 1732353876708398, "job": 31873, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [272117], "files_L3": [272135, 272136, 272056, 272057, 272058, 272059, 272060, 272061, 272062, 272063, 272064, 272065, 272066, 272067, 272068, 272069, 272070, 272071, 272072, 272073, 272074, 272075, 272137], "score": 0.998196, "input_data_size": 859893801, "oldest_snapshot_seqno": 4145465491} 2024/11/23-17:25:54.074430 139957592835648 (Original Log Time 2024/11/23-17:25:54.072797) [compaction_job.cc:2022] [default] [JOB 31873] Compacted 1@2 + 23@3 files to L3 => 844043458 bytes 2024/11/23-17:25:54.074432 139957592835648 (Original Log Time 2024/11/23-17:25:54.074349) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.9 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 792.9 +0.0 blob) out(804.9 +0.0 blob), read-write-amplify(59.9) write-amplify(29.7) OK, records in: 20917677, records dropped: 78206 output_compression: Snappy 2024/11/23-17:25:54.074436 139957592835648 (Original Log Time 2024/11/23-17:25:54.074394) EVENT_LOG_v1 {"time_micros": 1732353954074372, "cf": "default", "job": 31873, "event": "compaction_finished", "compaction_time_micros": 77229407, "compaction_time_cpu_micros": 33380390, "output_level": 3, "num_output_files": 23, "total_output_size": 844043458, "num_input_records": 20917677, "num_output_records": 20839471, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 612172454, "file_range_sync_nanos": 0, "file_fsync_nanos": 16317, "file_prepare_write_nanos": 67936, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-17:25:54.192356 139957592835648 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732353876,"executesMs":77507,"compactionJobId":31873,"attempt":0,"compactionInputRawBytes":3000608547,"compactionInputZipBytes":859893801,"compactionOutputRawBytes":0,"compactionOutputZipBytes":844043458,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a265943a688a31b1ea21122bf8ae728c-76c574e1461df379-00"}}, response = 2024/11/23-17:25:54.192435 139957592835648 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-31873/att-00: olev 3, work 77.507 s, result 7.849 ms, install 1.653 ms, input{raw 2.795 GiB zip 820.059 MiB}