2024/11/22-14:49:07.539509 140025179854400 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00: fork to child time = 0.009571 sec 2024/11/22-14:49:07.540412 140025179854400 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00 2024/11/22-14:49:07.547344 140025179854400 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00 2024/11/22-14:49:07.548449 140025179854400 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-14:49:07.548721 140025179854400 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-14:49:07.600465 140025179854400 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01460/att-00/MANIFEST-206347 2024/11/22-14:49:07.613703 140025179854400 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01460/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 208853, last_sequence is 3302518559, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 208706 2024/11/22-14:49:07.613710 140025179854400 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-14:49:07.613867 140025179854400 [version_set.cc:5721] Creating manifest 208853 2024/11/22-14:49:07.646286 140025179854400 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-14:49:07.680993 140025179854400 [compaction_job.cc:2437] [default] [JOB 1460] Compacting 1@2 + 22@3 files to L3, score 0.88, subcompactions 1 : 1 2024/11/22-14:49:07.681016 140025179854400 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 1204 Base level 2, inputs: [208848(13MB)], [208811(40MB) 208814(1717KB) 208779(39MB) 208780(40MB) 208800(39MB) 208801(39MB) 208802(39MB) 208799(39MB) 208803(39MB) 208804(39MB) 208805(39MB) 208806(39MB) 208807(39MB) 208808(39MB) 208813(39MB) 208815(39MB) 208818(39MB) 208816(39MB) 208819(39MB) 208820(39MB) 208821(39MB) 208817(10MB)] 2024/11/22-14:49:07.685744 140025179854400 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:49:07.685777 140025179854400 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:49:07.685805 140025179854400 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:49:07.685810 140025179854400 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:49:07.685822 140025179854400 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:49:08.085842 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst) /dev/shm/ToplingTemp/Topling-154368-yvpndO 2024/11/22-14:49:08.284124 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst): first pass time = 0.20's, 677.063'MB/sec 2024/11/22-14:49:09.543066 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst): old prealloc_size = 147639500, real_size = 40731312 2024/11/22-14:49:09.543203 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.01 avg-zval = 312.75 2024/11/22-14:49:09.543370 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:09.735239 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208854.sst): second pass time = 1.24's, 105.758'MB/sec, value only(98.1% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.19's, 220.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1905.674'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.242'MB/sec write SST data time = 0.06's, 722.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36833.556'MB/sec zip my value time = 1.24's, unzip length = 0.132'GB zip my value throughput = 105.758'MB/sec zip pipeline throughput = 162.252'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.01 avg-zval = 312.75 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1020.98 avg-zval = 312.75 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 73499 bytes = 238880 } seq expand size = 514493 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 7.0464 value = 3.2519 dict = 1.00 all = 3.1702 } Zip/UnZip{ index = 0.1419 value = 0.3075 dict = 1.00 all = 0.3154 } ---------------------------- total value len = 0.132145 GB avg = 1.021 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129429 Billion total entry num = 0.000129429 Billion write speed all = 0.006293986 MB/sec (with seq num) write speed all = 0.006245617 MB/sec (without seq num) 2024/11/22-14:49:09.774349 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208854: 129429 keys, 42338173 bytes, temperature: kWarm 2024/11/22-14:49:09.774425 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258149774375, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208854, "file_size": 42338173, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302491457, "table_properties": {"data_size": 40478368, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630020, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258148, "oldest_key_time": 0, "file_creation_time": 1732258148, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:09.814102 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst) /dev/shm/ToplingTemp/Topling-154368-XiyROK 2024/11/22-14:49:10.275846 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst): first pass time = 0.46's, 290.702'MB/sec 2024/11/22-14:49:11.445337 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst): old prealloc_size = 147639500, real_size = 40371904 2024/11/22-14:49:11.445464 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.95 2024/11/22-14:49:11.445553 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:11.694760 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208855.sst): second pass time = 1.17's, 112.613'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 167.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2172.716'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.179'MB/sec write SST data time = 0.07's, 611.994'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43717.167'MB/sec zip my value time = 1.17's, unzip length = 0.132'GB zip my value throughput = 112.613'MB/sec zip pipeline throughput = 161.043'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 309.95 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1017.17 avg-zval = 309.95 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 3237 bytes = 10528 } seq expand size = 22659 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 6.9897 value = 3.2811 dict = 1.00 all = 3.2148 } Zip/UnZip{ index = 0.1431 value = 0.3048 dict = 1.00 all = 0.3111 } ---------------------------- total value len = 0.263797 GB avg = 1.019 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258859 Billion total entry num = 0.000258859 Billion write speed all = 0.012563844 MB/sec (with seq num) write speed all = 0.012467113 MB/sec (without seq num) 2024/11/22-14:49:11.734981 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208855: 129430 keys, 41750733 bytes, temperature: kWarm 2024/11/22-14:49:11.735058 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258151735017, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208855, "file_size": 41750733, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302481642, "table_properties": {"data_size": 40117168, "index_size": 222208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629938, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258149, "oldest_key_time": 0, "file_creation_time": 1732258149, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:11.791349 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst) /dev/shm/ToplingTemp/Topling-154368-g9lSON 2024/11/22-14:49:12.268580 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst): first pass time = 0.48's, 281.273'MB/sec 2024/11/22-14:49:13.810376 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst): old prealloc_size = 147639500, real_size = 40612928 2024/11/22-14:49:13.810532 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.02 avg-zval = 311.85 2024/11/22-14:49:13.810612 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:14.147657 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208856.sst): second pass time = 1.54's, 85.472'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.34's, 124.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1662.067'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.157'MB/sec write SST data time = 0.13's, 321.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42006.922'MB/sec zip my value time = 1.54's, unzip length = 0.132'GB zip my value throughput = 85.472'MB/sec zip pipeline throughput = 118.626'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 311.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 311.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 146 bytes = 448 } seq expand size = 1022 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 7.0896 value = 3.2613 dict = 1.00 all = 3.1971 } Zip/UnZip{ index = 0.1411 value = 0.3066 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 0.395428 GB avg = 1.018 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388286 Billion total entry num = 0.000388286 Billion write speed all = 0.018831101 MB/sec (with seq num) write speed all = 0.018686022 MB/sec (without seq num) 2024/11/22-14:49:14.192691 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208856: 129427 keys, 41980730 bytes, temperature: kWarm 2024/11/22-14:49:14.192772 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258154192728, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208856, "file_size": 41980730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302490969, "table_properties": {"data_size": 40361328, "index_size": 219072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629816, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258151, "oldest_key_time": 0, "file_creation_time": 1732258151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:14.265974 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst) /dev/shm/ToplingTemp/Topling-154368-IgmHXM 2024/11/22-14:49:15.023371 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst): first pass time = 0.76's, 177.223'MB/sec 2024/11/22-14:49:16.137174 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst): old prealloc_size = 147639500, real_size = 40529056 2024/11/22-14:49:16.137289 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.19 2024/11/22-14:49:16.137340 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:16.294761 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208857.sst): second pass time = 1.11's, 118.241'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2371.706'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.306'MB/sec write SST data time = 0.05's, 812.731'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37289.031'MB/sec zip my value time = 1.11's, unzip length = 0.132'GB zip my value throughput = 118.241'MB/sec zip pipeline throughput = 124.133'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 311.19 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 311.19 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 7.0981 value = 3.2681 dict = 1.00 all = 3.2036 } Zip/UnZip{ index = 0.1409 value = 0.3060 dict = 1.00 all = 0.3122 } ---------------------------- total value len = 0.527058 GB avg = 1.018 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517718 Billion total entry num = 0.000517718 Billion write speed all = 0.025097248 MB/sec (with seq num) write speed all = 0.024903828 MB/sec (without seq num) 2024/11/22-14:49:16.338521 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208857: 129432 keys, 41896650 bytes, temperature: kWarm 2024/11/22-14:49:16.338598 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258156338555, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208857, "file_size": 41896650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302474225, "table_properties": {"data_size": 40277712, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629922, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258154, "oldest_key_time": 0, "file_creation_time": 1732258154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:16.407611 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst) /dev/shm/ToplingTemp/Topling-154368-3AMOvL 2024/11/22-14:49:16.916501 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst): first pass time = 0.51's, 263.771'MB/sec 2024/11/22-14:49:18.016298 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst): old prealloc_size = 147639500, real_size = 40356736 2024/11/22-14:49:18.016445 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.87 2024/11/22-14:49:18.016509 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:18.246382 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208858.sst): second pass time = 1.10's, 119.759'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 181.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1774.491'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.197'MB/sec write SST data time = 0.07's, 634.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34724.625'MB/sec zip my value time = 1.10's, unzip length = 0.132'GB zip my value throughput = 119.759'MB/sec zip pipeline throughput = 132.502'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 9 bytes = 32 } seq expand size = 63 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.658688 GB avg = 1.018 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647146 Billion total entry num = 0.000647146 Billion write speed all = 0.031362403 MB/sec (with seq num) write speed all = 0.031120650 MB/sec (without seq num) 2024/11/22-14:49:18.284729 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208858: 129428 keys, 41724507 bytes, temperature: kWarm 2024/11/22-14:49:18.284815 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258158284763, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208858, "file_size": 41724507, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302396464, "table_properties": {"data_size": 40105392, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629591, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258156, "oldest_key_time": 0, "file_creation_time": 1732258156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:18.358368 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst) /dev/shm/ToplingTemp/Topling-154368-JtSF6N 2024/11/22-14:49:19.071693 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst): first pass time = 0.71's, 188.178'MB/sec 2024/11/22-14:49:20.276237 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst): old prealloc_size = 147639500, real_size = 40357744 2024/11/22-14:49:20.276370 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.87 2024/11/22-14:49:20.276455 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:20.640420 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208859.sst): second pass time = 1.20's, 109.496'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.36's, 114.584'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1746.295'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.109'MB/sec write SST data time = 0.07's, 620.179'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44638.866'MB/sec zip my value time = 1.20's, unzip length = 0.132'GB zip my value throughput = 109.496'MB/sec zip pipeline throughput = 109.496'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 15 bytes = 48 } seq expand size = 105 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.790318 GB avg = 1.018 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776574 Billion total entry num = 0.000776574 Billion write speed all = 0.037625659 MB/sec (with seq num) write speed all = 0.037335589 MB/sec (without seq num) 2024/11/22-14:49:20.677867 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208859: 129428 keys, 41725744 bytes, temperature: kWarm 2024/11/22-14:49:20.677926 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258160677896, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208859, "file_size": 41725744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302474364, "table_properties": {"data_size": 40106400, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629980, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258158, "oldest_key_time": 0, "file_creation_time": 1732258158, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:20.717699 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst) /dev/shm/ToplingTemp/Topling-154368-lASHNO 2024/11/22-14:49:21.440637 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst): first pass time = 0.72's, 185.667'MB/sec 2024/11/22-14:49:22.479719 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst): old prealloc_size = 147639500, real_size = 40354528 2024/11/22-14:49:22.479845 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.85 2024/11/22-14:49:22.479901 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:22.638997 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208860.sst): second pass time = 1.04's, 126.874'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2093.733'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.309'MB/sec write SST data time = 0.05's, 765.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34251.727'MB/sec zip my value time = 1.04's, unzip length = 0.132'GB zip my value throughput = 126.874'MB/sec zip pipeline throughput = 126.874'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 21 bytes = 64 } seq expand size = 147 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.921947 GB avg = 1.018 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000906001 Billion total entry num = 0.000906001 Billion write speed all = 0.043888289 MB/sec (with seq num) write speed all = 0.043549906 MB/sec (without seq num) 2024/11/22-14:49:22.685236 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208860: 129427 keys, 41722339 bytes, temperature: kWarm 2024/11/22-14:49:22.685314 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258162685271, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208860, "file_size": 41722339, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302405919, "table_properties": {"data_size": 40103184, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629197, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258160, "oldest_key_time": 0, "file_creation_time": 1732258160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:22.755342 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst) /dev/shm/ToplingTemp/Topling-154368-JS8JVO 2024/11/22-14:49:23.671665 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst): first pass time = 0.92's, 146.485'MB/sec 2024/11/22-14:49:25.335299 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst): old prealloc_size = 147639500, real_size = 40353328 2024/11/22-14:49:25.335451 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.98 avg-zval = 309.83 2024/11/22-14:49:25.335528 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:25.606831 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208861.sst): second pass time = 1.66's, 79.301'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 153.669'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1917.840'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.168'MB/sec write SST data time = 0.08's, 530.070'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.377'MB, throughput = 34440.519'MB/sec zip my value time = 1.66's, unzip length = 0.132'GB zip my value throughput = 79.301'MB/sec zip pipeline throughput = 150.812'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.83 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.98 avg-zval = 309.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2824 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.053577 GB avg = 1.018 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035433 Billion total entry num = 0.001035433 Billion write speed all = 0.050147516 MB/sec (with seq num) write speed all = 0.049760845 MB/sec (without seq num) 2024/11/22-14:49:25.661409 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208861: 129432 keys, 41720586 bytes, temperature: kWarm 2024/11/22-14:49:25.661477 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258165661438, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208861, "file_size": 41720586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302484941, "table_properties": {"data_size": 40101984, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629871, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258162, "oldest_key_time": 0, "file_creation_time": 1732258162, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:25.707586 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst) /dev/shm/ToplingTemp/Topling-154368-XfNm2N 2024/11/22-14:49:26.167916 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst): first pass time = 0.46's, 291.597'MB/sec 2024/11/22-14:49:26.974682 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst): old prealloc_size = 147639500, real_size = 40352000 2024/11/22-14:49:26.974816 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.82 2024/11/22-14:49:26.974868 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:27.132813 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208862.sst): second pass time = 0.81's, 163.263'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.868'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2012.916'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 793.071'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44443.087'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.263'MB/sec zip pipeline throughput = 163.263'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.82 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 23 bytes = 80 } seq expand size = 161 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2825 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.185207 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164866 Billion total entry num = 0.001164866 Billion write speed all = 0.056408804 MB/sec (with seq num) write speed all = 0.055973828 MB/sec (without seq num) 2024/11/22-14:49:27.188614 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208862: 129433 keys, 41720028 bytes, temperature: kWarm 2024/11/22-14:49:27.188672 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258167188640, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208862, "file_size": 41720028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302420287, "table_properties": {"data_size": 40100656, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629766, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258165, "oldest_key_time": 0, "file_creation_time": 1732258165, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:27.236841 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst) /dev/shm/ToplingTemp/Topling-154368-XvybRO 2024/11/22-14:49:27.694765 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst): first pass time = 0.46's, 293.131'MB/sec 2024/11/22-14:49:28.530610 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst): old prealloc_size = 147639500, real_size = 40356256 2024/11/22-14:49:28.530724 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.86 2024/11/22-14:49:28.530789 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:28.684311 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208863.sst): second pass time = 0.84's, 157.585'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.505'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2401.352'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.314'MB/sec write SST data time = 0.05's, 826.303'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45061.321'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 157.585'MB/sec zip pipeline throughput = 157.585'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.316837 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015532 GB avg = 0.012 KB total ukey num = 0.001294295 Billion total entry num = 0.001294295 Billion write speed all = 0.062669110 MB/sec (with seq num) write speed all = 0.062185840 MB/sec (without seq num) 2024/11/22-14:49:28.722641 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208863: 129429 keys, 41723823 bytes, temperature: kWarm 2024/11/22-14:49:28.722699 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258168722669, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208863, "file_size": 41723823, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302476628, "table_properties": {"data_size": 40104912, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629568, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258167, "oldest_key_time": 0, "file_creation_time": 1732258167, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:28.758944 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst) /dev/shm/ToplingTemp/Topling-154368-zwcGFN 2024/11/22-14:49:29.425532 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst): first pass time = 0.67's, 201.363'MB/sec 2024/11/22-14:49:30.224507 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst): old prealloc_size = 147639500, real_size = 40353648 2024/11/22-14:49:30.224620 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:49:30.224673 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:30.376660 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208864.sst): second pass time = 0.80's, 164.865'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.229'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2426.517'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.313'MB/sec write SST data time = 0.05's, 856.224'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45836.194'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.865'MB/sec zip pipeline throughput = 164.865'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 20 bytes = 64 } seq expand size = 140 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.448466 GB avg = 1.017 KB (by entry num) total key len = 0.028475 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423726 Billion total entry num = 0.001423726 Billion write speed all = 0.068928065 MB/sec (with seq num) write speed all = 0.068396509 MB/sec (without seq num) 2024/11/22-14:49:30.413761 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208864: 129431 keys, 41721259 bytes, temperature: kWarm 2024/11/22-14:49:30.413829 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258170413800, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208864, "file_size": 41721259, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302312384, "table_properties": {"data_size": 40102304, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629680, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258168, "oldest_key_time": 0, "file_creation_time": 1732258168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:30.450923 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst) /dev/shm/ToplingTemp/Topling-154368-QJL6XO 2024/11/22-14:49:30.884726 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst): first pass time = 0.43's, 309.430'MB/sec 2024/11/22-14:49:31.688506 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst): old prealloc_size = 147639500, real_size = 40356672 2024/11/22-14:49:31.688623 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.87 2024/11/22-14:49:31.688675 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:31.840342 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208865.sst): second pass time = 0.80's, 163.880'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.827'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2338.877'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 875.778'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42445.916'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 163.880'MB/sec zip pipeline throughput = 163.880'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.580096 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553153 Billion total entry num = 0.001553153 Billion write speed all = 0.075186827 MB/sec (with seq num) write speed all = 0.074606988 MB/sec (without seq num) 2024/11/22-14:49:31.877371 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208865: 129427 keys, 41724565 bytes, temperature: kWarm 2024/11/22-14:49:31.877436 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258171877401, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208865, "file_size": 41724565, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302434577, "table_properties": {"data_size": 40105328, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629655, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258170, "oldest_key_time": 0, "file_creation_time": 1732258170, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:31.911713 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst) /dev/shm/ToplingTemp/Topling-154368-wK82XK 2024/11/22-14:49:32.371978 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst): first pass time = 0.46's, 291.637'MB/sec 2024/11/22-14:49:33.251930 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst): old prealloc_size = 147639500, real_size = 40354400 2024/11/22-14:49:33.252065 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:49:33.252137 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:33.492344 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208866.sst): second pass time = 0.88's, 149.704'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 173.549'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1983.808'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.181'MB/sec write SST data time = 0.06's, 679.573'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 41658.634'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.704'MB/sec zip pipeline throughput = 175.213'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.711725 GB avg = 1.017 KB (by entry num) total key len = 0.033652 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682583 Billion total entry num = 0.001682583 Billion write speed all = 0.081443998 MB/sec (with seq num) write speed all = 0.080815887 MB/sec (without seq num) 2024/11/22-14:49:33.529156 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208866: 129430 keys, 41721903 bytes, temperature: kWarm 2024/11/22-14:49:33.529226 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258173529188, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208866, "file_size": 41721903, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302479657, "table_properties": {"data_size": 40103056, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629155, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258171, "oldest_key_time": 0, "file_creation_time": 1732258171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:33.570459 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst) /dev/shm/ToplingTemp/Topling-154368-dfM5wK 2024/11/22-14:49:34.026970 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst): first pass time = 0.46's, 294.039'MB/sec 2024/11/22-14:49:34.820522 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst): old prealloc_size = 147639500, real_size = 40348656 2024/11/22-14:49:34.820636 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.79 2024/11/22-14:49:34.820697 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:34.972192 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208867.sst): second pass time = 0.79's, 165.991'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 275.094'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2422.005'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.310'MB/sec write SST data time = 0.05's, 885.886'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45906.692'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 165.991'MB/sec zip pipeline throughput = 165.991'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.79 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2828 dict = 1.00 all = 3.2174 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.843355 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001812017 Billion total entry num = 0.001812017 Billion write speed all = 0.087700947 MB/sec (with seq num) write speed all = 0.087024565 MB/sec (without seq num) 2024/11/22-14:49:35.015930 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208867: 129434 keys, 41716504 bytes, temperature: kWarm 2024/11/22-14:49:35.015989 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258175015958, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208867, "file_size": 41716504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302418000, "table_properties": {"data_size": 40097312, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629884, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258173, "oldest_key_time": 0, "file_creation_time": 1732258173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:35.053820 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst) /dev/shm/ToplingTemp/Topling-154368-XNn3WO 2024/11/22-14:49:35.487105 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst): first pass time = 0.43's, 309.797'MB/sec 2024/11/22-14:49:36.285714 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst): old prealloc_size = 147639500, real_size = 40357440 2024/11/22-14:49:36.285843 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.87 2024/11/22-14:49:36.285898 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:36.435400 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208868.sst): second pass time = 0.80's, 164.952'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 278.776'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2074.361'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.315'MB/sec write SST data time = 0.05's, 890.297'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42881.052'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.952'MB/sec zip pipeline throughput = 164.952'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 21 bytes = 64 } seq expand size = 147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.974986 GB avg = 1.017 KB (by entry num) total key len = 0.038829 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941444 Billion total entry num = 0.001941444 Billion write speed all = 0.093957102 MB/sec (with seq num) write speed all = 0.093232458 MB/sec (without seq num) 2024/11/22-14:49:36.474434 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208868: 129427 keys, 41725396 bytes, temperature: kWarm 2024/11/22-14:49:36.474495 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258176474464, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208868, "file_size": 41725396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302480559, "table_properties": {"data_size": 40106096, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629926, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258175, "oldest_key_time": 0, "file_creation_time": 1732258175, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:36.520810 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst) /dev/shm/ToplingTemp/Topling-154368-iPcShN 2024/11/22-14:49:37.056881 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst): first pass time = 0.54's, 250.396'MB/sec 2024/11/22-14:49:37.872662 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst): old prealloc_size = 147639500, real_size = 40357264 2024/11/22-14:49:37.872777 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.87 2024/11/22-14:49:37.872844 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:38.023472 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208869.sst): second pass time = 0.82's, 161.472'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 276.699'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2130.052'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.313'MB/sec write SST data time = 0.05's, 884.741'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43771.442'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.472'MB/sec zip pipeline throughput = 161.472'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.106616 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070873 Billion total entry num = 0.002070873 Billion write speed all = 0.100211827 MB/sec (with seq num) write speed all = 0.099438930 MB/sec (without seq num) 2024/11/22-14:49:38.059244 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208869: 129429 keys, 41724932 bytes, temperature: kWarm 2024/11/22-14:49:38.059304 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258178059274, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208869, "file_size": 41724932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302254815, "table_properties": {"data_size": 40105920, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630118, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258176, "oldest_key_time": 0, "file_creation_time": 1732258176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:38.097359 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst) /dev/shm/ToplingTemp/Topling-154368-yxIoBL 2024/11/22-14:49:38.568542 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst): first pass time = 0.47's, 284.880'MB/sec 2024/11/22-14:49:39.372411 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst): old prealloc_size = 147639500, real_size = 40359120 2024/11/22-14:49:39.372523 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.89 2024/11/22-14:49:39.372575 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:39.524458 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208870.sst): second pass time = 0.80's, 163.857'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2426.221'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.312'MB/sec write SST data time = 0.05's, 866.568'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45113.581'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 163.857'MB/sec zip pipeline throughput = 163.857'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.89 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2819 dict = 1.00 all = 3.2166 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.238245 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026404 GB avg = 0.012 KB total ukey num = 0.002200299 Billion total entry num = 0.002200299 Billion write speed all = 0.106466023 MB/sec (with seq num) write speed all = 0.105644879 MB/sec (without seq num) 2024/11/22-14:49:39.567838 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208870: 129426 keys, 41727128 bytes, temperature: kWarm 2024/11/22-14:49:39.567897 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258179567867, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208870, "file_size": 41727128, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302287498, "table_properties": {"data_size": 40107776, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629410, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258178, "oldest_key_time": 0, "file_creation_time": 1732258178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:39.601469 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst) /dev/shm/ToplingTemp/Topling-154368-2vJhBO 2024/11/22-14:49:40.209928 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst): first pass time = 0.61's, 220.603'MB/sec 2024/11/22-14:49:41.051486 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst): old prealloc_size = 147639500, real_size = 40348240 2024/11/22-14:49:41.051601 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129436 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.95 avg-zval = 309.78 2024/11/22-14:49:41.051652 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:41.203131 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208871.sst): second pass time = 0.84's, 156.520'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 275.129'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2404.334'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.312'MB/sec write SST data time = 0.05's, 872.579'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35501.687'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.520'MB/sec zip pipeline throughput = 156.520'MB/sec entries = 129436 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.78 usrkeys = 129436 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.78 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2828 dict = 1.00 all = 3.2174 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.369875 GB avg = 1.017 KB (by entry num) total key len = 0.046595 GB avg = 0.020 KB total ukey len = 0.027957 GB avg = 0.012 KB total ukey num = 0.002329735 Billion total entry num = 0.002329735 Billion write speed all = 0.112718437 MB/sec (with seq num) write speed all = 0.111849057 MB/sec (without seq num) 2024/11/22-14:49:41.243277 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208871: 129436 keys, 41716407 bytes, temperature: kWarm 2024/11/22-14:49:41.243339 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258181243308, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208871, "file_size": 41716407, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302490353, "table_properties": {"data_size": 40096896, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588720, "raw_average_key_size": 20, "raw_value_size": 131629830, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258179, "oldest_key_time": 0, "file_creation_time": 1732258179, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:41.278990 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst) /dev/shm/ToplingTemp/Topling-154368-DorJyO 2024/11/22-14:49:41.850381 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst): first pass time = 0.57's, 234.915'MB/sec 2024/11/22-14:49:42.637610 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst): old prealloc_size = 147639500, real_size = 40354496 2024/11/22-14:49:42.637723 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:49:42.637775 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:42.789300 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208872.sst): second pass time = 0.79's, 167.450'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 275.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2419.835'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 879.204'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45389.592'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.450'MB/sec zip pipeline throughput = 167.450'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 18 bytes = 64 } seq expand size = 126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.501505 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459165 Billion total entry num = 0.002459165 Billion write speed all = 0.118970355 MB/sec (with seq num) write speed all = 0.118052743 MB/sec (without seq num) 2024/11/22-14:49:42.827012 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208872: 129430 keys, 41722283 bytes, temperature: kWarm 2024/11/22-14:49:42.827081 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258182827042, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208872, "file_size": 41722283, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302272125, "table_properties": {"data_size": 40103152, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629253, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258181, "oldest_key_time": 0, "file_creation_time": 1732258181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:42.863897 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst) /dev/shm/ToplingTemp/Topling-154368-UhkapL 2024/11/22-14:49:43.300606 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst): first pass time = 0.44's, 307.368'MB/sec 2024/11/22-14:49:44.080206 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst): old prealloc_size = 147639500, real_size = 40358672 2024/11/22-14:49:44.080322 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.89 2024/11/22-14:49:44.080371 140025179854400 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:49:44.232271 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208873.sst): second pass time = 0.78's, 168.962'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2422.621'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 870.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44964.747'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.962'MB/sec zip pipeline throughput = 168.962'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2819 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.633134 GB avg = 1.017 KB (by entry num) total key len = 0.051772 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588591 Billion total entry num = 0.002588591 Billion write speed all = 0.125222199 MB/sec (with seq num) write speed all = 0.124256358 MB/sec (without seq num) 2024/11/22-14:49:44.269270 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208873: 129426 keys, 41725955 bytes, temperature: kWarm 2024/11/22-14:49:44.269333 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258184269302, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208873, "file_size": 41725955, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302459631, "table_properties": {"data_size": 40107328, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629694, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258182, "oldest_key_time": 0, "file_creation_time": 1732258182, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:44.306217 140025179854400 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208874.sst) /dev/shm/ToplingTemp/Topling-154368-77z47K 2024/11/22-14:49:44.435223 140025179854400 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208874.sst): first pass time = 0.13's, 316.460'MB/sec 2024/11/22-14:49:44.753370 140025179854400 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208874.sst): old prealloc_size = 147639500, real_size = 12312680 2024/11/22-14:49:44.753417 140025179854400 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208874.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 39354 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0400 GB zip-val = 0.0123 GB avg-val =1017.05 avg-zval = 312.61 2024/11/22-14:49:44.800936 140025179854400 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00/208874.sst): second pass time = 0.32's, 126.016'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.05's, 267.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.259'MB/sec (index lex order gen) rebuild zvType time = 0.03's, 0.299'MB/sec write SST data time = 0.01's, 870.920'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.394'MB, throughput = 13573.783'MB/sec zip my value time = 0.32's, unzip length = 0.040'GB zip my value throughput = 126.016'MB/sec zip pipeline throughput = 126.016'MB/sec entries = 39354 avg-key = 20.00 avg-zkey = 0.00 avg-val = 1017.05 avg-zval = 312.61 usrkeys = 39354 avg-key = 12.00 avg-zkey = 0.00 avg-val = 1017.06 avg-zval = 312.61 TagRS{ kinds = RS_Key0_TagN bytes = 6240 } TagArray{ size = 6 bytes = 32 } seq expand size = 42 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0005 GB value = 0.0400 GB dict = 0.39 MB all = 0.0408 GB } __ZipSize{ index = 0.0000 GB value = 0.0123 GB dict = 0.39 MB all = 0.0127 GB } UnZip/Zip{ index =2459.6250 value = 3.2534 dict = 1.00 all = 3.2126 } Zip/UnZip{ index = 0.0004 value = 0.3074 dict = 1.00 all = 0.3113 } ---------------------------- total value len = 2.673160 GB avg = 1.017 KB (by entry num) total key len = 0.052559 GB avg = 0.020 KB total ukey len = 0.031535 GB avg = 0.012 KB total ukey num = 0.002627945 Billion total entry num = 0.002627945 Billion write speed all = 0.127122284 MB/sec (with seq num) write speed all = 0.126141786 MB/sec (without seq num) 2024/11/22-14:49:44.816222 140025179854400 [compaction_job.cc:1948] [default] [JOB 1460] Generated table #208874: 39354 keys, 12703905 bytes, temperature: kWarm 2024/11/22-14:49:44.816282 140025179854400 EVENT_LOG_v1 {"time_micros": 1732258184816252, "cf_name": "default", "job": 1460, "event": "table_file_creation", "file_number": 208874, "file_size": 12703905, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3302204113, "table_properties": {"data_size": 12302512, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 787080, "raw_average_key_size": 20, "raw_value_size": 40025164, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 39354, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732258184, "oldest_key_time": 0, "file_creation_time": 1732258184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 208874, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:49:44.831343 140025179854400 [compaction_job.cc:703] job-01460: subcompact[0], size: 0.848234 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01460/att-00 [208854,208855,208856,208857,208858,208859,208860,208861,208862,208863,208864,208865,208866,208867,208868,208869,208870,208871,208872,208873,208874] 2024/11/22-14:49:44.972176 140025179854400 (Original Log Time 2024/11/22-14:49:07.681071) EVENT_LOG_v1 {"time_micros": 1732258147681030, "job": 1460, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [208848], "files_L3": [208811, 208814, 208779, 208780, 208800, 208801, 208802, 208799, 208803, 208804, 208805, 208806, 208807, 208808, 208813, 208815, 208818, 208816, 208819, 208820, 208821, 208817], "score": 0.878759, "input_data_size": 861544114, "oldest_snapshot_seqno": 3302518543} 2024/11/22-14:49:44.972179 140025179854400 (Original Log Time 2024/11/22-14:49:44.971128) [compaction_job.cc:2022] [default] [JOB 1460] Compacted 1@2 + 22@3 files to L3 => 848233550 bytes 2024/11/22-14:49:44.972181 140025179854400 (Original Log Time 2024/11/22-14:49:44.972102) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 23.2 rd, 22.8 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(13.1, 808.6 +0.0 blob) out(808.9 +0.0 blob), read-write-amplify(124.9) write-amplify(62.0) OK, records in: 2641069, records dropped: 13124 output_compression: Snappy 2024/11/22-14:49:44.972184 140025179854400 (Original Log Time 2024/11/22-14:49:44.972150) EVENT_LOG_v1 {"time_micros": 1732258184972127, "cf": "default", "job": 1460, "event": "compaction_finished", "compaction_time_micros": 37150282, "compaction_time_cpu_micros": 7884331, "output_level": 3, "num_output_files": 21, "total_output_size": 848233550, "num_input_records": 2641069, "num_output_records": 2627945, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 712451399, "file_range_sync_nanos": 0, "file_fsync_nanos": 19762, "file_prepare_write_nanos": 53846, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/22-14:49:45.108166 140025179854400 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732258147,"executesMs":37426,"compactionJobId":1460,"attempt":0,"compactionInputRawBytes":2738788784,"compactionInputZipBytes":861544114,"compactionOutputRawBytes":0,"compactionOutputZipBytes":848233550,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-047b2076ff8bab4f938ce732121724c4-6fe916ccf698417c-00"}}, response = 2024/11/22-14:49:45.108238 140025179854400 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01460/att-00: olev 3, work 37.426 s, result 4.986 ms, install 1.078 ms, input{raw 2.551 GiB zip 821.632 MiB}