2024/11/22-14:54:48.321974 140024886109760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00: fork to child time = 0.029835 sec 2024/11/22-14:54:48.323028 140024886109760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00 2024/11/22-14:54:48.332173 140024886109760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00 2024/11/22-14:54:48.333619 140024886109760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-14:54:48.334016 140024886109760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-14:54:48.398524 140024886109760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01697/att-00/MANIFEST-206347 2024/11/22-14:54:48.433975 140024886109760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01697/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 209242, last_sequence is 3307157931, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 209074 2024/11/22-14:54:48.433989 140024886109760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-14:54:48.434278 140024886109760 [version_set.cc:5721] Creating manifest 209242 2024/11/22-14:54:48.467970 140024886109760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-14:54:48.490282 140024886109760 [compaction_job.cc:2437] [default] [JOB 1697] Compacting 1@2 + 22@3 files to L3, score 0.85, subcompactions 1 : 1 2024/11/22-14:54:48.490315 140024886109760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 1393 Base level 2, inputs: [209227(12MB)], [209167(40MB) 209169(1628KB) 209132(39MB) 209133(40MB) 209162(39MB) 209161(39MB) 209164(39MB) 209163(39MB) 209170(39MB) 209171(39MB) 209172(39MB) 209173(39MB) 209174(39MB) 209175(39MB) 209182(39MB) 209183(39MB) 209184(39MB) 209185(39MB) 209186(39MB) 209188(39MB) 209189(39MB) 209187(14MB)] 2024/11/22-14:54:48.495521 140024886109760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:54:48.495564 140024886109760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:54:48.495592 140024886109760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-14:54:48.495601 140024886109760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:54:48.495617 140024886109760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-14:54:48.939045 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst) /dev/shm/ToplingTemp/Topling-157851-MrH369 2024/11/22-14:54:49.142005 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst): first pass time = 0.20's, 661.449'MB/sec 2024/11/22-14:54:50.021196 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst): old prealloc_size = 147639500, real_size = 40749360 2024/11/22-14:54:50.021331 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.02 avg-zval = 312.89 2024/11/22-14:54:50.021518 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:50.245334 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209243.sst): second pass time = 0.88's, 150.096'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 189.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1963.442'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.196'MB/sec write SST data time = 0.06's, 720.798'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38766.005'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.096'MB/sec zip pipeline throughput = 165.140'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1017.02 avg-zval = 312.89 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1021.14 avg-zval = 312.89 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 76224 bytes = 247744 } seq expand size = 533568 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 7.0463 value = 3.2504 dict = 1.00 all = 3.1681 } Zip/UnZip{ index = 0.1419 value = 0.3077 dict = 1.00 all = 0.3156 } ---------------------------- total value len = 0.132163 GB avg = 1.021 KB (by entry num) total key len = 0.002589 GB avg = 0.020 KB total ukey len = 0.001553 GB avg = 0.012 KB total ukey num = 0.000129427 Billion total entry num = 0.000129427 Billion write speed all = 0.006196284 MB/sec (with seq num) write speed all = 0.006148673 MB/sec (without seq num) 2024/11/22-14:54:50.281770 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209243: 129427 keys, 42365024 bytes, temperature: kWarm 2024/11/22-14:54:50.281861 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258490281809, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209243, "file_size": 42365024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307060085, "table_properties": {"data_size": 40496416, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629414, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258488, "oldest_key_time": 0, "file_creation_time": 1732258488, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:50.321866 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst) /dev/shm/ToplingTemp/Topling-157851-Bm3gZa 2024/11/22-14:54:50.757152 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst): first pass time = 0.44's, 308.375'MB/sec 2024/11/22-14:54:51.601831 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst): old prealloc_size = 147639500, real_size = 40375120 2024/11/22-14:54:51.601971 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.97 2024/11/22-14:54:51.602044 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:51.849521 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209244.sst): second pass time = 0.84's, 156.148'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 168.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1913.987'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.180'MB/sec write SST data time = 0.07's, 613.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46349.963'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.148'MB/sec zip pipeline throughput = 156.148'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 309.97 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1017.19 avg-zval = 309.97 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 3822 bytes = 12432 } seq expand size = 26754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0418 GB } UnZip/Zip{ index = 6.9757 value = 3.2809 dict = 1.00 all = 3.2144 } Zip/UnZip{ index = 0.1434 value = 0.3048 dict = 1.00 all = 0.3111 } ---------------------------- total value len = 0.263819 GB avg = 1.019 KB (by entry num) total key len = 0.005177 GB avg = 0.020 KB total ukey len = 0.003106 GB avg = 0.012 KB total ukey num = 0.000258858 Billion total entry num = 0.000258858 Billion write speed all = 0.012368361 MB/sec (with seq num) write speed all = 0.012273144 MB/sec (without seq num) 2024/11/22-14:54:51.896914 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209244: 129431 keys, 41755142 bytes, temperature: kWarm 2024/11/22-14:54:51.896972 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258491896942, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209244, "file_size": 41755142, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307040625, "table_properties": {"data_size": 40119936, "index_size": 222656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629546, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258490, "oldest_key_time": 0, "file_creation_time": 1732258490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:51.932638 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst) /dev/shm/ToplingTemp/Topling-157851-fUhMi8 2024/11/22-14:54:52.466068 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst): first pass time = 0.53's, 251.632'MB/sec 2024/11/22-14:54:53.284677 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst): old prealloc_size = 147639500, real_size = 40567088 2024/11/22-14:54:53.284800 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.49 2024/11/22-14:54:53.284850 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:53.438418 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209245.sst): second pass time = 0.82's, 160.906'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 272.790'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2208.297'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.306'MB/sec write SST data time = 0.05's, 878.786'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47577.250'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.906'MB/sec zip pipeline throughput = 160.906'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 311.49 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.04 avg-zval = 311.49 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 382 bytes = 1216 } seq expand size = 2674 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 7.0834 value = 3.2650 dict = 1.00 all = 3.2006 } Zip/UnZip{ index = 0.1412 value = 0.3063 dict = 1.00 all = 0.3124 } ---------------------------- total value len = 0.395451 GB avg = 1.018 KB (by entry num) total key len = 0.007766 GB avg = 0.020 KB total ukey len = 0.004659 GB avg = 0.012 KB total ukey num = 0.000388285 Billion total entry num = 0.000388285 Billion write speed all = 0.018538425 MB/sec (with seq num) write speed all = 0.018395609 MB/sec (without seq num) 2024/11/22-14:54:53.475242 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209245: 129427 keys, 41935757 bytes, temperature: kWarm 2024/11/22-14:54:53.475303 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258493475271, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209245, "file_size": 41935757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307053561, "table_properties": {"data_size": 40315296, "index_size": 219264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629424, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258491, "oldest_key_time": 0, "file_creation_time": 1732258491, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:53.510937 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst) /dev/shm/ToplingTemp/Topling-157851-URNhT8 2024/11/22-14:54:54.070442 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst): first pass time = 0.56's, 239.903'MB/sec 2024/11/22-14:54:54.877876 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst): old prealloc_size = 147639500, real_size = 40577616 2024/11/22-14:54:54.877995 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.57 2024/11/22-14:54:54.878051 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:55.030453 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209246.sst): second pass time = 0.81's, 163.146'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 274.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2352.228'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.308'MB/sec write SST data time = 0.05's, 882.097'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42326.533'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.146'MB/sec zip pipeline throughput = 163.146'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 311.57 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 311.57 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 7.0980 value = 3.2641 dict = 1.00 all = 3.1999 } Zip/UnZip{ index = 0.1409 value = 0.3064 dict = 1.00 all = 0.3125 } ---------------------------- total value len = 0.527081 GB avg = 1.018 KB (by entry num) total key len = 0.010354 GB avg = 0.020 KB total ukey len = 0.006213 GB avg = 0.012 KB total ukey num = 0.000517715 Billion total entry num = 0.000517715 Billion write speed all = 0.024707466 MB/sec (with seq num) write speed all = 0.024517059 MB/sec (without seq num) 2024/11/22-14:54:55.067084 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209246: 129430 keys, 41945003 bytes, temperature: kWarm 2024/11/22-14:54:55.067146 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258495067114, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209246, "file_size": 41945003, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307003641, "table_properties": {"data_size": 40326272, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629360, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258493, "oldest_key_time": 0, "file_creation_time": 1732258493, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:55.104857 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst) /dev/shm/ToplingTemp/Topling-157851-RXb8i8 2024/11/22-14:54:55.797162 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst): first pass time = 0.69's, 193.882'MB/sec 2024/11/22-14:54:56.621109 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst): old prealloc_size = 147639500, real_size = 40355648 2024/11/22-14:54:56.621226 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.01 avg-zval = 309.86 2024/11/22-14:54:56.621283 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:56.775337 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209247.sst): second pass time = 0.82's, 159.877'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2325.801'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 856.727'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36954.612'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 159.877'MB/sec zip pipeline throughput = 159.877'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 12 bytes = 48 } seq expand size = 84 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2822 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.658710 GB avg = 1.018 KB (by entry num) total key len = 0.012943 GB avg = 0.020 KB total ukey len = 0.007766 GB avg = 0.012 KB total ukey num = 0.000647143 Billion total entry num = 0.000647143 Billion write speed all = 0.030875377 MB/sec (with seq num) write speed all = 0.030637388 MB/sec (without seq num) 2024/11/22-14:54:56.811335 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209247: 129428 keys, 41723851 bytes, temperature: kWarm 2024/11/22-14:54:56.811399 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258496811366, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209247, "file_size": 41723851, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307020052, "table_properties": {"data_size": 40104304, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629231, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258495, "oldest_key_time": 0, "file_creation_time": 1732258495, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:56.850984 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst) /dev/shm/ToplingTemp/Topling-157851-a64M77 2024/11/22-14:54:57.493232 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst): first pass time = 0.64's, 208.997'MB/sec 2024/11/22-14:54:58.445065 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst): old prealloc_size = 147639500, real_size = 40354736 2024/11/22-14:54:58.445188 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.85 2024/11/22-14:54:58.445239 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:54:58.599300 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209248.sst): second pass time = 0.95's, 138.410'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 270.534'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2260.636'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.308'MB/sec write SST data time = 0.05's, 851.682'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 43595.971'MB/sec zip my value time = 0.95's, unzip length = 0.132'GB zip my value throughput = 138.410'MB/sec zip pipeline throughput = 138.410'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.01 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.790340 GB avg = 1.018 KB (by entry num) total key len = 0.015531 GB avg = 0.020 KB total ukey len = 0.009319 GB avg = 0.012 KB total ukey num = 0.000776572 Billion total entry num = 0.000776572 Billion write speed all = 0.037042196 MB/sec (with seq num) write speed all = 0.036756633 MB/sec (without seq num) 2024/11/22-14:54:58.637207 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209248: 129429 keys, 41723190 bytes, temperature: kWarm 2024/11/22-14:54:58.637268 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258498637237, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209248, "file_size": 41723190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307024111, "table_properties": {"data_size": 40103392, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629884, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258496, "oldest_key_time": 0, "file_creation_time": 1732258496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:54:58.673101 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst) /dev/shm/ToplingTemp/Topling-157851-YtQH0a 2024/11/22-14:54:59.350108 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst): first pass time = 0.68's, 198.265'MB/sec 2024/11/22-14:55:00.152687 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst): old prealloc_size = 147639500, real_size = 40357664 2024/11/22-14:55:00.152832 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.88 2024/11/22-14:55:00.152903 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:00.324251 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209249.sst): second pass time = 0.80's, 164.148'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 243.210'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1909.039'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 831.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46158.482'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.148'MB/sec zip pipeline throughput = 164.148'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 0.921970 GB avg = 1.018 KB (by entry num) total key len = 0.018120 GB avg = 0.020 KB total ukey len = 0.010872 GB avg = 0.012 KB total ukey num = 0.000905999 Billion total entry num = 0.000905999 Billion write speed all = 0.043208182 MB/sec (with seq num) write speed all = 0.042875052 MB/sec (without seq num) 2024/11/22-14:55:00.361286 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209249: 129427 keys, 41725314 bytes, temperature: kWarm 2024/11/22-14:55:00.361351 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258500361319, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209249, "file_size": 41725314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307004845, "table_properties": {"data_size": 40106320, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630017, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258498, "oldest_key_time": 0, "file_creation_time": 1732258498, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:00.398380 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst) /dev/shm/ToplingTemp/Topling-157851-abwkJ8 2024/11/22-14:55:01.036336 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst): first pass time = 0.64's, 210.401'MB/sec 2024/11/22-14:55:01.907159 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst): old prealloc_size = 147639500, real_size = 40354288 2024/11/22-14:55:01.907289 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:55:01.907357 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:02.135171 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209250.sst): second pass time = 0.87's, 151.297'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 182.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2160.186'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.191'MB/sec write SST data time = 0.06's, 711.916'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50074.684'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 151.297'MB/sec zip pipeline throughput = 169.637'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 16 bytes = 64 } seq expand size = 112 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.053600 GB avg = 1.018 KB (by entry num) total key len = 0.020709 GB avg = 0.020 KB total ukey len = 0.012425 GB avg = 0.012 KB total ukey num = 0.001035429 Billion total entry num = 0.001035429 Billion write speed all = 0.049372966 MB/sec (with seq num) write speed all = 0.048992277 MB/sec (without seq num) 2024/11/22-14:55:02.172325 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209250: 129430 keys, 41721918 bytes, temperature: kWarm 2024/11/22-14:55:02.172388 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258502172356, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209250, "file_size": 41721918, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307016518, "table_properties": {"data_size": 40102944, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629314, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258500, "oldest_key_time": 0, "file_creation_time": 1732258500, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:02.216421 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst) /dev/shm/ToplingTemp/Topling-157851-uxczHa 2024/11/22-14:55:02.668767 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst): first pass time = 0.45's, 296.752'MB/sec 2024/11/22-14:55:03.463360 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst): old prealloc_size = 147639500, real_size = 40351248 2024/11/22-14:55:03.463489 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.96 avg-zval = 309.81 2024/11/22-14:55:03.463545 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:03.616319 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209251.sst): second pass time = 0.79's, 165.785'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 272.773'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2085.611'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 876.347'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40306.509'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 165.785'MB/sec zip pipeline throughput = 165.785'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.81 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.96 avg-zval = 309.81 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 17 bytes = 64 } seq expand size = 119 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2825 dict = 1.00 all = 3.2172 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.185229 GB avg = 1.017 KB (by entry num) total key len = 0.023297 GB avg = 0.020 KB total ukey len = 0.013978 GB avg = 0.012 KB total ukey num = 0.001164864 Billion total entry num = 0.001164864 Billion write speed all = 0.055537584 MB/sec (with seq num) write speed all = 0.055109336 MB/sec (without seq num) 2024/11/22-14:55:03.656794 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209251: 129435 keys, 41719393 bytes, temperature: kWarm 2024/11/22-14:55:03.656849 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258503656819, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209251, "file_size": 41719393, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306931148, "table_properties": {"data_size": 40099904, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629608, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258502, "oldest_key_time": 0, "file_creation_time": 1732258502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:03.692728 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst) /dev/shm/ToplingTemp/Topling-157851-1tliza 2024/11/22-14:55:04.142408 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst): first pass time = 0.45's, 298.497'MB/sec 2024/11/22-14:55:04.930276 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst): old prealloc_size = 147639500, real_size = 40356352 2024/11/22-14:55:04.930394 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.00 avg-zval = 309.86 2024/11/22-14:55:04.930461 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:05.084043 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209252.sst): second pass time = 0.79's, 167.194'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 271.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1984.078'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.310'MB/sec write SST data time = 0.05's, 845.472'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49073.173'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.194'MB/sec zip pipeline throughput = 167.194'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2821 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.316859 GB avg = 1.017 KB (by entry num) total key len = 0.025886 GB avg = 0.020 KB total ukey len = 0.015532 GB avg = 0.012 KB total ukey num = 0.001294294 Billion total entry num = 0.001294294 Billion write speed all = 0.061701401 MB/sec (with seq num) write speed all = 0.061225601 MB/sec (without seq num) 2024/11/22-14:55:05.123893 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209252: 129430 keys, 41724282 bytes, temperature: kWarm 2024/11/22-14:55:05.123956 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258505123924, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209252, "file_size": 41724282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306925684, "table_properties": {"data_size": 40105008, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629721, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258503, "oldest_key_time": 0, "file_creation_time": 1732258503, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:05.161969 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst) /dev/shm/ToplingTemp/Topling-157851-9o4Z29 2024/11/22-14:55:05.705738 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst): first pass time = 0.54's, 246.848'MB/sec 2024/11/22-14:55:07.922383 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst): old prealloc_size = 147639500, real_size = 40354272 2024/11/22-14:55:07.922521 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:55:07.922587 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:08.082755 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209253.sst): second pass time = 2.22's, 59.420'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2002.306'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.294'MB/sec write SST data time = 0.05's, 828.060'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43269.882'MB/sec zip my value time = 2.22's, unzip length = 0.132'GB zip my value throughput = 59.420'MB/sec zip pipeline throughput = 106.034'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2823 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.448489 GB avg = 1.017 KB (by entry num) total key len = 0.028474 GB avg = 0.020 KB total ukey len = 0.017085 GB avg = 0.012 KB total ukey num = 0.001423724 Billion total entry num = 0.001423724 Billion write speed all = 0.067859586 MB/sec (with seq num) write speed all = 0.067336278 MB/sec (without seq num) 2024/11/22-14:55:08.124028 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209253: 129430 keys, 41722188 bytes, temperature: kWarm 2024/11/22-14:55:08.124110 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258508124064, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209253, "file_size": 41722188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307040522, "table_properties": {"data_size": 40102928, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629156, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258505, "oldest_key_time": 0, "file_creation_time": 1732258505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209253, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:08.176214 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst) /dev/shm/ToplingTemp/Topling-157851-g1AJ7a 2024/11/22-14:55:08.931974 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst): first pass time = 0.76's, 177.605'MB/sec 2024/11/22-14:55:09.798117 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst): old prealloc_size = 147639500, real_size = 40357840 2024/11/22-14:55:09.798234 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.88 2024/11/22-14:55:09.798292 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:09.959692 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209254.sst): second pass time = 0.86's, 152.251'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2339.027'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.292'MB/sec write SST data time = 0.05's, 821.837'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46043.735'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.251'MB/sec zip pipeline throughput = 152.251'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.88 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0978 value = 3.2820 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.580119 GB avg = 1.017 KB (by entry num) total key len = 0.031063 GB avg = 0.020 KB total ukey len = 0.018638 GB avg = 0.012 KB total ukey num = 0.001553150 Billion total entry num = 0.001553150 Billion write speed all = 0.074019928 MB/sec (with seq num) write speed all = 0.073449097 MB/sec (without seq num) 2024/11/22-14:55:09.997219 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209254: 129426 keys, 41725672 bytes, temperature: kWarm 2024/11/22-14:55:09.997286 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258509997255, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209254, "file_size": 41725672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306931088, "table_properties": {"data_size": 40106496, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630078, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258508, "oldest_key_time": 0, "file_creation_time": 1732258508, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209254, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:10.039093 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst) /dev/shm/ToplingTemp/Topling-157851-EDGvk8 2024/11/22-14:55:10.794588 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst): first pass time = 0.76's, 177.666'MB/sec 2024/11/22-14:55:12.319998 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst): old prealloc_size = 147639500, real_size = 40355072 2024/11/22-14:55:12.320115 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/22-14:55:12.320172 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:12.476644 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209255.sst): second pass time = 1.52's, 86.325'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2351.975'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.307'MB/sec write SST data time = 0.05's, 813.855'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 29256.200'MB/sec zip my value time = 1.52's, unzip length = 0.132'GB zip my value throughput = 86.325'MB/sec zip pipeline throughput = 112.836'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 14 bytes = 48 } seq expand size = 98 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.711748 GB avg = 1.017 KB (by entry num) total key len = 0.033652 GB avg = 0.020 KB total ukey len = 0.020191 GB avg = 0.012 KB total ukey num = 0.001682581 Billion total entry num = 0.001682581 Billion write speed all = 0.080176836 MB/sec (with seq num) write speed all = 0.079558507 MB/sec (without seq num) 2024/11/22-14:55:12.513513 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209255: 129431 keys, 41722681 bytes, temperature: kWarm 2024/11/22-14:55:12.513575 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258512513544, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209255, "file_size": 41722681, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307036730, "table_properties": {"data_size": 40103728, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629670, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258510, "oldest_key_time": 0, "file_creation_time": 1732258510, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209255, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:12.560037 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst) /dev/shm/ToplingTemp/Topling-157851-KAcd67 2024/11/22-14:55:13.201599 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst): first pass time = 0.64's, 209.220'MB/sec 2024/11/22-14:55:14.303247 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst): old prealloc_size = 147639500, real_size = 40352352 2024/11/22-14:55:14.303372 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.97 avg-zval = 309.82 2024/11/22-14:55:14.303421 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:14.457949 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209256.sst): second pass time = 1.10's, 119.555'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 269.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2313.066'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.311'MB/sec write SST data time = 0.05's, 827.651'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39157.896'MB/sec zip my value time = 1.10's, unzip length = 0.132'GB zip my value throughput = 119.555'MB/sec zip pipeline throughput = 127.905'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.82 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.97 avg-zval = 309.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0982 value = 3.2825 dict = 1.00 all = 3.2171 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 1.843378 GB avg = 1.017 KB (by entry num) total key len = 0.036240 GB avg = 0.020 KB total ukey len = 0.021744 GB avg = 0.012 KB total ukey num = 0.001812014 Billion total entry num = 0.001812014 Billion write speed all = 0.086334453 MB/sec (with seq num) write speed all = 0.085668619 MB/sec (without seq num) 2024/11/22-14:55:14.495951 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209256: 129433 keys, 41720202 bytes, temperature: kWarm 2024/11/22-14:55:14.496013 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258514495980, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209256, "file_size": 41720202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307015943, "table_properties": {"data_size": 40101008, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629807, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258512, "oldest_key_time": 0, "file_creation_time": 1732258512, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209256, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:14.535425 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst) /dev/shm/ToplingTemp/Topling-157851-3Er0t9 2024/11/22-14:55:15.419980 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst): first pass time = 0.88's, 151.742'MB/sec 2024/11/22-14:55:16.592334 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst): old prealloc_size = 147639500, real_size = 40356960 2024/11/22-14:55:16.592474 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.87 2024/11/22-14:55:16.592541 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:16.763213 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209257.sst): second pass time = 1.17's, 112.427'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 244.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1941.821'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 813.988'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36184.725'MB/sec zip my value time = 1.17's, unzip length = 0.132'GB zip my value throughput = 112.427'MB/sec zip pipeline throughput = 127.493'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.87 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 11 bytes = 48 } seq expand size = 77 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2821 dict = 1.00 all = 3.2167 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 1.975009 GB avg = 1.017 KB (by entry num) total key len = 0.038829 GB avg = 0.020 KB total ukey len = 0.023297 GB avg = 0.012 KB total ukey num = 0.001941441 Billion total entry num = 0.001941441 Billion write speed all = 0.092489581 MB/sec (with seq num) write speed all = 0.091776264 MB/sec (without seq num) 2024/11/22-14:55:16.800886 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209257: 129427 keys, 41724927 bytes, temperature: kWarm 2024/11/22-14:55:16.800949 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258516800915, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209257, "file_size": 41724927, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306914538, "table_properties": {"data_size": 40105616, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630106, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258514, "oldest_key_time": 0, "file_creation_time": 1732258514, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:16.836891 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst) /dev/shm/ToplingTemp/Topling-157851-WbN3R9 2024/11/22-14:55:17.366005 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst): first pass time = 0.53's, 253.685'MB/sec 2024/11/22-14:55:18.600502 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst): old prealloc_size = 147639500, real_size = 40355104 2024/11/22-14:55:18.600639 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.85 2024/11/22-14:55:18.600714 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:18.779630 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209258.sst): second pass time = 1.23's, 106.687'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 232.931'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1937.454'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.254'MB/sec write SST data time = 0.05's, 811.562'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36697.816'MB/sec zip my value time = 1.23's, unzip length = 0.132'GB zip my value throughput = 106.687'MB/sec zip pipeline throughput = 199.215'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.85 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.00 avg-zval = 309.85 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0980 value = 3.2822 dict = 1.00 all = 3.2169 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.106638 GB avg = 1.017 KB (by entry num) total key len = 0.041417 GB avg = 0.020 KB total ukey len = 0.024850 GB avg = 0.012 KB total ukey num = 0.002070871 Billion total entry num = 0.002070871 Billion write speed all = 0.098644693 MB/sec (with seq num) write speed all = 0.097883891 MB/sec (without seq num) 2024/11/22-14:55:18.822413 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209258: 129430 keys, 41722834 bytes, temperature: kWarm 2024/11/22-14:55:18.822476 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258518822443, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209258, "file_size": 41722834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306803436, "table_properties": {"data_size": 40103760, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629536, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258516, "oldest_key_time": 0, "file_creation_time": 1732258516, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:18.859419 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst) /dev/shm/ToplingTemp/Topling-157851-fgt4v8 2024/11/22-14:55:19.426510 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst): first pass time = 0.57's, 236.694'MB/sec 2024/11/22-14:55:20.259160 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst): old prealloc_size = 147639500, real_size = 40358416 2024/11/22-14:55:20.259294 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.03 avg-zval = 309.89 2024/11/22-14:55:20.259361 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:20.414767 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209259.sst): second pass time = 0.83's, 158.336'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.170'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2122.737'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.305'MB/sec write SST data time = 0.05's, 846.993'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47286.067'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.336'MB/sec zip pipeline throughput = 158.336'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.03 avg-zval = 309.89 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 19 bytes = 64 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0977 value = 3.2820 dict = 1.00 all = 3.2166 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.238268 GB avg = 1.017 KB (by entry num) total key len = 0.044006 GB avg = 0.020 KB total ukey len = 0.026404 GB avg = 0.012 KB total ukey num = 0.002200296 Billion total entry num = 0.002200296 Billion write speed all = 0.104800492 MB/sec (with seq num) write speed all = 0.103992203 MB/sec (without seq num) 2024/11/22-14:55:20.460276 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209259: 129425 keys, 41726075 bytes, temperature: kWarm 2024/11/22-14:55:20.460346 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258520460306, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209259, "file_size": 41726075, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306991421, "table_properties": {"data_size": 40107072, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629470, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258518, "oldest_key_time": 0, "file_creation_time": 1732258518, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:20.502031 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst) /dev/shm/ToplingTemp/Topling-157851-xneSHa 2024/11/22-14:55:20.967369 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst): first pass time = 0.47's, 288.456'MB/sec 2024/11/22-14:55:21.851226 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst): old prealloc_size = 147639500, real_size = 40349552 2024/11/22-14:55:21.851370 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129436 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.95 avg-zval = 309.79 2024/11/22-14:55:21.851441 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:22.080573 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209260.sst): second pass time = 0.88's, 149.054'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 181.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1895.808'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.200'MB/sec write SST data time = 0.07's, 620.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38611.222'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.054'MB/sec zip pipeline throughput = 153.718'MB/sec entries = 129436 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.79 usrkeys = 129436 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.95 avg-zval = 309.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 10 bytes = 48 } seq expand size = 70 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0983 value = 3.2827 dict = 1.00 all = 3.2173 } Zip/UnZip{ index = 0.1409 value = 0.3046 dict = 1.00 all = 0.3108 } ---------------------------- total value len = 2.369897 GB avg = 1.017 KB (by entry num) total key len = 0.046595 GB avg = 0.020 KB total ukey len = 0.027957 GB avg = 0.012 KB total ukey num = 0.002329732 Billion total entry num = 0.002329732 Billion write speed all = 0.110955213 MB/sec (with seq num) write speed all = 0.110099440 MB/sec (without seq num) 2024/11/22-14:55:22.124247 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209260: 129436 keys, 41717014 bytes, temperature: kWarm 2024/11/22-14:55:22.124326 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258522124283, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209260, "file_size": 41717014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307002120, "table_properties": {"data_size": 40098208, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588720, "raw_average_key_size": 20, "raw_value_size": 131629348, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258520, "oldest_key_time": 0, "file_creation_time": 1732258520, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:22.171592 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst) /dev/shm/ToplingTemp/Topling-157851-xJeQ1a 2024/11/22-14:55:22.656121 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst): first pass time = 0.48's, 277.039'MB/sec 2024/11/22-14:55:23.485265 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst): old prealloc_size = 147639500, real_size = 40354176 2024/11/22-14:55:23.485394 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1016.99 avg-zval = 309.84 2024/11/22-14:55:23.485458 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:23.644928 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209261.sst): second pass time = 0.83's, 159.014'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.325'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2058.825'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.296'MB/sec write SST data time = 0.05's, 830.531'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36469.306'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.014'MB/sec zip pipeline throughput = 159.014'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1016.99 avg-zval = 309.84 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0981 value = 3.2823 dict = 1.00 all = 3.2170 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.501527 GB avg = 1.017 KB (by entry num) total key len = 0.049183 GB avg = 0.020 KB total ukey len = 0.029510 GB avg = 0.012 KB total ukey num = 0.002459163 Billion total entry num = 0.002459163 Billion write speed all = 0.117109559 MB/sec (with seq num) write speed all = 0.116206308 MB/sec (without seq num) 2024/11/22-14:55:23.684037 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209261: 129431 keys, 41722183 bytes, temperature: kWarm 2024/11/22-14:55:23.684098 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258523684067, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209261, "file_size": 41722183, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306989412, "table_properties": {"data_size": 40102832, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629916, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258522, "oldest_key_time": 0, "file_creation_time": 1732258522, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:23.738081 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst) /dev/shm/ToplingTemp/Topling-157851-PEfCc8 2024/11/22-14:55:24.197205 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst): first pass time = 0.46's, 292.362'MB/sec 2024/11/22-14:55:25.086082 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst): old prealloc_size = 147639500, real_size = 40356160 2024/11/22-14:55:25.086212 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.69 raw-val = 0.1316 GB zip-val = 0.0401 GB avg-val =1017.02 avg-zval = 309.86 2024/11/22-14:55:25.086261 140024886109760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-14:55:25.238557 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209262.sst): second pass time = 0.89's, 148.307'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 273.668'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2294.462'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.308'MB/sec write SST data time = 0.05's, 882.955'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45758.093'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 148.307'MB/sec zip pipeline throughput = 148.307'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.86 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.69 avg-val = 1017.02 avg-zval = 309.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 13 bytes = 48 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0401 GB dict = 1.38 MB all = 0.0417 GB } UnZip/Zip{ index = 7.0979 value = 3.2822 dict = 1.00 all = 3.2168 } Zip/UnZip{ index = 0.1409 value = 0.3047 dict = 1.00 all = 0.3109 } ---------------------------- total value len = 2.633157 GB avg = 1.017 KB (by entry num) total key len = 0.051772 GB avg = 0.020 KB total ukey len = 0.031063 GB avg = 0.012 KB total ukey num = 0.002588590 Billion total entry num = 0.002588590 Billion write speed all = 0.123262859 MB/sec (with seq num) write speed all = 0.122312139 MB/sec (without seq num) 2024/11/22-14:55:25.274849 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209262: 129427 keys, 41724107 bytes, temperature: kWarm 2024/11/22-14:55:25.274910 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258525274878, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209262, "file_size": 41724107, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3306946707, "table_properties": {"data_size": 40104816, "index_size": 218816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630058, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732258523, "oldest_key_time": 0, "file_creation_time": 1732258523, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:25.312216 140024886109760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209263.sst) /dev/shm/ToplingTemp/Topling-157851-5A8rR9 2024/11/22-14:55:25.513516 140024886109760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209263.sst): first pass time = 0.20's, 269.276'MB/sec 2024/11/22-14:55:26.070698 140024886109760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209263.sst): old prealloc_size = 147639500, real_size = 16299336 2024/11/22-14:55:26.070756 140024886109760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209263.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 52258 prefix = 4 raw-key = 0.0004 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0531 GB zip-val = 0.0163 GB avg-val =1017.06 avg-zval = 311.64 2024/11/22-14:55:26.156614 140024886109760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00/209263.sst): second pass time = 0.56's, 95.506'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.09's, 195.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 6.491'MB/sec (index lex order gen) rebuild zvType time = 0.07's, 0.196'MB/sec write SST data time = 0.02's, 877.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.539'MB, throughput = 13063.828'MB/sec zip my value time = 0.56's, unzip length = 0.053'GB zip my value throughput = 95.506'MB/sec zip pipeline throughput = 180.966'MB/sec entries = 52258 avg-key = 20.00 avg-zkey = 0.00 avg-val = 1017.06 avg-zval = 311.64 usrkeys = 52258 avg-key = 12.00 avg-zkey = 0.00 avg-val = 1017.06 avg-zval = 311.64 TagRS{ kinds = RS_Key0_TagN bytes = 8280 } TagArray{ size = 1 bytes = 16 } seq expand size = 7 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0006 GB value = 0.0531 GB dict = 0.54 MB all = 0.0542 GB } __ZipSize{ index = 0.0000 GB value = 0.0163 GB dict = 0.54 MB all = 0.0168 GB } UnZip/Zip{ index =3266.1250 value = 3.2635 dict = 1.00 all = 3.2192 } Zip/UnZip{ index = 0.0003 value = 0.3064 dict = 1.00 all = 0.3106 } ---------------------------- total value len = 2.686307 GB avg = 1.017 KB (by entry num) total key len = 0.052817 GB avg = 0.020 KB total ukey len = 0.031690 GB avg = 0.012 KB total ukey num = 0.002640848 Billion total entry num = 0.002640848 Billion write speed all = 0.125745593 MB/sec (with seq num) write speed all = 0.124775721 MB/sec (without seq num) 2024/11/22-14:55:26.177898 140024886109760 [compaction_job.cc:1948] [default] [JOB 1697] Generated table #209263: 52258 keys, 16834660 bytes, temperature: kWarm 2024/11/22-14:55:26.177960 140024886109760 EVENT_LOG_v1 {"time_micros": 1732258526177929, "cf_name": "default", "job": 1697, "event": "table_file_creation", "file_number": 209263, "file_size": 16834660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3307043902, "table_properties": {"data_size": 16285904, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1045160, "raw_average_key_size": 20, "raw_value_size": 53149525, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 52258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1732258525, "oldest_key_time": 0, "file_creation_time": 1732258525, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 209263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-14:55:26.194082 140024886109760 [compaction_job.cc:703] job-01697: subcompact[0], size: 0.852401 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-01697/att-00 [209243,209244,209245,209246,209247,209248,209249,209250,209251,209252,209253,209254,209255,209256,209257,209258,209259,209260,209261,209262,209263] 2024/11/22-14:55:26.330083 140024886109760 (Original Log Time 2024/11/22-14:54:48.490399) EVENT_LOG_v1 {"time_micros": 1732258488490337, "job": 1697, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [209227], "files_L3": [209167, 209169, 209132, 209133, 209162, 209161, 209164, 209163, 209170, 209171, 209172, 209173, 209174, 209175, 209182, 209183, 209184, 209185, 209186, 209188, 209189, 209187], "score": 0.853158, "input_data_size": 864967075, "oldest_snapshot_seqno": 3307157868} 2024/11/22-14:55:26.330086 140024886109760 (Original Log Time 2024/11/22-14:55:26.329062) [compaction_job.cc:2022] [default] [JOB 1697] Compacted 1@2 + 22@3 files to L3 => 852401417 bytes 2024/11/22-14:55:26.330087 140024886109760 (Original Log Time 2024/11/22-14:55:26.330010) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.9 rd, 22.6 wr, level 3, files in(1, 22) out(21 +0 blob) MB in(12.3, 812.6 +0.0 blob) out(812.9 +0.0 blob), read-write-amplify(132.7) write-amplify(65.9) OK, records in: 2653228, records dropped: 12380 output_compression: Snappy 2024/11/22-14:55:26.330091 140024886109760 (Original Log Time 2024/11/22-14:55:26.330058) EVENT_LOG_v1 {"time_micros": 1732258526330035, "cf": "default", "job": 1697, "event": "compaction_finished", "compaction_time_micros": 37703692, "compaction_time_cpu_micros": 8105792, "output_level": 3, "num_output_files": 21, "total_output_size": 852401417, "num_input_records": 2653228, "num_output_records": 2640848, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 588370879, "file_range_sync_nanos": 0, "file_fsync_nanos": 19230, "file_prepare_write_nanos": 44711, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/22-14:55:26.454773 140024886109760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732258488,"executesMs":38001,"compactionJobId":1697,"attempt":0,"compactionInputRawBytes":2751398354,"compactionInputZipBytes":864967075,"compactionOutputRawBytes":0,"compactionOutputZipBytes":852401417,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-cbaad8b01a6cee115a1eabcd9f288744-278c945566aab359-00"}}, response = 2024/11/22-14:55:26.454852 140024886109760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-01697/att-00: olev 3, work 38.002 s, result 5.000 ms, install 1.063 ms, input{raw 2.562 GiB zip 824.897 MiB}