2024/11/24-05:48:36.647277 139957441766976 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00: fork to child time = 0.007909 sec 2024/11/24-05:48:36.648152 139957441766976 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00 2024/11/24-05:48:36.658149 139957441766976 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00 2024/11/24-05:48:36.659164 139957441766976 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:48:36.659456 139957441766976 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:48:36.728223 139957441766976 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42409/att-00/MANIFEST-206347 2024/11/24-05:48:36.754202 139957441766976 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42409/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294015, last_sequence is 4466480335, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 293865 2024/11/24-05:48:36.754219 139957441766976 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:48:36.754421 139957441766976 [version_set.cc:5721] Creating manifest 294015 2024/11/24-05:48:36.787049 139957441766976 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:48:36.807431 139957441766976 [compaction_job.cc:2437] [default] [JOB 42409] Compacting 1@2 + 23@3 files to L3, score 0.78, subcompactions 1 : 1 2024/11/24-05:48:36.807464 139957441766976 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31796 Base level 2, inputs: [294011(27MB)], [293948(33MB) 293949(33MB) 293950(35MB) 293951(33MB) 293952(35MB) 293953(33MB) 293954(33MB) 293957(35MB) 293958(33MB) 293959(35MB) 293960(34MB) 293961(33MB) 293962(35MB) 293963(31MB) 293964(30MB) 293965(35MB) 293966(32MB) 293967(32MB) 293968(33MB) 293969(32MB) 293970(35MB) 293971(33MB) 293972(35MB)] 2024/11/24-05:48:36.816153 139957441766976 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:48:36.816195 139957441766976 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:48:36.816225 139957441766976 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:48:36.816233 139957441766976 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:48:36.816250 139957441766976 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:48:37.452311 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst) /dev/shm/ToplingTemp/Topling-283069-Jkh0xx 2024/11/24-05:48:38.496593 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst): first pass time = 1.04's, 128.532'MB/sec 2024/11/24-05:48:40.152280 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst): old prealloc_size = 147639500, real_size = 30690192 2024/11/24-05:48:40.152988 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927422 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.72 avg-zval = 30.93 2024/11/24-05:48:40.153833 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:41.312411 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294016.sst): second pass time = 1.65's, 65.423'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 29.887'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2637.855'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.241'MB/sec write SST data time = 0.20's, 175.662'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34687.321'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.423'MB/sec zip pipeline throughput = 65.423'MB/sec entries = 927422 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.72 avg-zval = 30.93 usrkeys = 927422 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.72 avg-zval = 30.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927422 bytes = 3130064 } seq expand size = 6491954 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4443 value = 3.7741 dict = 1.00 all = 3.8710 } Zip/UnZip{ index = 0.0957 value = 0.2650 dict = 1.00 all = 0.2583 } ---------------------------- total value len = 2251.440315 GB avg = 0.200 KB (by entry num) total key len = 306.908930 GB avg = 0.027 KB total ukey len = 237.373276 GB avg = 0.015 KB total ukey num = 15.739588585 Billion total entry num = 11.280902533 Billion write speed all = 34.016486325 MB/sec (with seq num) write speed all = 32.816535482 MB/sec (without seq num) 2024/11/24-05:48:41.360868 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294016: 927422 keys, 34672806 bytes, temperature: kWarm 2024/11/24-05:48:41.360960 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398521360904, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294016, "file_size": 34672806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4362885758, "largest_seqno": 4466453678, "table_properties": {"data_size": 28682240, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967816, "raw_average_key_size": 28, "raw_value_size": 108250000, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927422, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398517, "oldest_key_time": 0, "file_creation_time": 1732398517, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294016, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:41.402935 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst) /dev/shm/ToplingTemp/Topling-283069-mpJhow 2024/11/24-05:48:42.626035 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst): first pass time = 1.22's, 109.739'MB/sec 2024/11/24-05:48:44.023527 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst): old prealloc_size = 147639500, real_size = 31531312 2024/11/24-05:48:44.024098 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919449 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.98 avg-zval = 32.13 2024/11/24-05:48:44.024890 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:45.837637 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294017.sst): second pass time = 1.39's, 78.179'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.81's, 19.630'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3211.029'MB/sec (index lex order gen) rebuild zvType time = 1.56's, 0.147'MB/sec write SST data time = 0.25's, 139.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 37643.182'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.179'MB/sec zip pipeline throughput = 78.179'MB/sec entries = 919449 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.98 avg-zval = 32.13 usrkeys = 919449 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.98 avg-zval = 32.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919449 bytes = 3218080 } seq expand size = 6436143 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4430 value = 3.6720 dict = 1.00 all = 3.7691 } Zip/UnZip{ index = 0.0958 value = 0.2723 dict = 1.00 all = 0.2653 } ---------------------------- total value len = 2251.555225 GB avg = 0.200 KB (by entry num) total key len = 306.934675 GB avg = 0.027 KB total ukey len = 237.391665 GB avg = 0.015 KB total ukey num = 15.740508034 Billion total entry num = 11.281821982 Billion write speed all = 34.016309793 MB/sec (with seq num) write speed all = 32.816333348 MB/sec (without seq num) 2024/11/24-05:48:45.892100 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294017: 919449 keys, 35610404 bytes, temperature: kWarm 2024/11/24-05:48:45.892165 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398525892132, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294017, "file_size": 35610404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4466454164, "table_properties": {"data_size": 29540384, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25744572, "raw_average_key_size": 28, "raw_value_size": 108473214, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398521, "oldest_key_time": 0, "file_creation_time": 1732398521, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294017, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:45.939924 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst) /dev/shm/ToplingTemp/Topling-283069-93tz0w 2024/11/24-05:48:46.967212 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst): first pass time = 1.03's, 130.658'MB/sec 2024/11/24-05:48:48.681298 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst): old prealloc_size = 147639500, real_size = 33023648 2024/11/24-05:48:48.681862 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927560 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.70 avg-zval = 33.45 2024/11/24-05:48:48.682659 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:49.640656 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294018.sst): second pass time = 1.71's, 63.181'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.698'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.912'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.13's, 294.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35222.675'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.181'MB/sec zip pipeline throughput = 77.265'MB/sec entries = 927560 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 33.45 usrkeys = 927560 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927560 bytes = 3246480 } seq expand size = 6492920 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5145 value = 3.4887 dict = 1.00 all = 3.6153 } Zip/UnZip{ index = 0.0951 value = 0.2866 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 2251.669964 GB avg = 0.200 KB (by entry num) total key len = 306.960647 GB avg = 0.027 KB total ukey len = 237.410216 GB avg = 0.015 KB total ukey num = 15.741435594 Billion total entry num = 11.282749542 Billion write speed all = 34.016460629 MB/sec (with seq num) write speed all = 32.816446202 MB/sec (without seq num) 2024/11/24-05:48:49.686709 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294018: 927560 keys, 37124574 bytes, temperature: kWarm 2024/11/24-05:48:49.686781 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398529686743, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294018, "file_size": 37124574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4466454055, "table_properties": {"data_size": 31027216, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971680, "raw_average_key_size": 28, "raw_value_size": 108246180, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927560, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398525, "oldest_key_time": 0, "file_creation_time": 1732398525, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294018, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:49.725317 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst) /dev/shm/ToplingTemp/Topling-283069-fPyupy 2024/11/24-05:48:50.722084 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst): first pass time = 1.00's, 134.658'MB/sec 2024/11/24-05:48:52.080559 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst): old prealloc_size = 147639500, real_size = 31377344 2024/11/24-05:48:52.081163 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-05:48:52.081937 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:52.985383 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294019.sst): second pass time = 1.36's, 79.897'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 39.185'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3055.207'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.295'MB/sec write SST data time = 0.13's, 281.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27119.826'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.897'MB/sec zip pipeline throughput = 79.897'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3764 value = 3.6928 dict = 1.00 all = 3.7857 } Zip/UnZip{ index = 0.0964 value = 0.2708 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2251.784876 GB avg = 0.200 KB (by entry num) total key len = 306.986387 GB avg = 0.027 KB total ukey len = 237.428602 GB avg = 0.015 KB total ukey num = 15.742354894 Billion total entry num = 11.283668842 Billion write speed all = 34.016817939 MB/sec (with seq num) write speed all = 32.816759100 MB/sec (without seq num) 2024/11/24-05:48:53.029177 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294019: 919300 keys, 35454066 bytes, temperature: kWarm 2024/11/24-05:48:53.029247 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398533029210, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294019, "file_size": 35454066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4312276494, "largest_seqno": 4463492790, "table_properties": {"data_size": 29375408, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398529, "oldest_key_time": 0, "file_creation_time": 1732398529, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294019, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:53.068452 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst) /dev/shm/ToplingTemp/Topling-283069-r4lKfy 2024/11/24-05:48:53.989873 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst): first pass time = 0.92's, 145.669'MB/sec 2024/11/24-05:48:55.344274 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst): old prealloc_size = 147639500, real_size = 33243552 2024/11/24-05:48:55.344856 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927622 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.69 avg-zval = 33.68 2024/11/24-05:48:55.345674 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:56.204813 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294020.sst): second pass time = 1.35's, 79.999'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3191.926'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 291.151'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36974.115'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.999'MB/sec zip pipeline throughput = 79.999'MB/sec entries = 927622 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.68 usrkeys = 927622 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927622 bytes = 3246688 } seq expand size = 6493354 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5129 value = 3.4642 dict = 1.00 all = 3.5941 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 2251.899614 GB avg = 0.200 KB (by entry num) total key len = 307.012360 GB avg = 0.027 KB total ukey len = 237.447154 GB avg = 0.015 KB total ukey num = 15.743282516 Billion total entry num = 11.284596464 Billion write speed all = 34.017232651 MB/sec (with seq num) write speed all = 32.817126521 MB/sec (without seq num) 2024/11/24-05:48:56.251606 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294020: 927622 keys, 37344010 bytes, temperature: kWarm 2024/11/24-05:48:56.251675 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398536251638, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294020, "file_size": 37344010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4466454284, "table_properties": {"data_size": 31246736, "index_size": 1764736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973416, "raw_average_key_size": 28, "raw_value_size": 108244364, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927622, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398533, "oldest_key_time": 0, "file_creation_time": 1732398533, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294020, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:56.290772 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst) /dev/shm/ToplingTemp/Topling-283069-udfvdw 2024/11/24-05:48:57.239345 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst): first pass time = 0.95's, 141.500'MB/sec 2024/11/24-05:48:58.558286 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst): old prealloc_size = 147639500, real_size = 30745024 2024/11/24-05:48:58.558838 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927386 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.73 avg-zval = 30.99 2024/11/24-05:48:58.559595 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:48:59.386667 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294021.sst): second pass time = 1.32's, 82.129'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.924'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3433.426'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 268.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35301.010'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.129'MB/sec zip pipeline throughput = 82.129'MB/sec entries = 927386 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.73 avg-zval = 30.99 usrkeys = 927386 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.73 avg-zval = 30.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927386 bytes = 3129936 } seq expand size = 6491702 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4439 value = 3.7669 dict = 1.00 all = 3.8649 } Zip/UnZip{ index = 0.0957 value = 0.2655 dict = 1.00 all = 0.2587 } ---------------------------- total value len = 2252.014356 GB avg = 0.200 KB (by entry num) total key len = 307.038327 GB avg = 0.027 KB total ukey len = 237.465702 GB avg = 0.015 KB total ukey num = 15.744209902 Billion total entry num = 11.285523850 Billion write speed all = 34.017664305 MB/sec (with seq num) write speed all = 32.817510312 MB/sec (without seq num) 2024/11/24-05:48:59.430925 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294021: 927386 keys, 34727578 bytes, temperature: kWarm 2024/11/24-05:48:59.431012 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398539430959, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294021, "file_size": 34727578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4362937430, "largest_seqno": 4466453724, "table_properties": {"data_size": 28737072, "index_size": 1775936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966808, "raw_average_key_size": 28, "raw_value_size": 108250920, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927386, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398536, "oldest_key_time": 0, "file_creation_time": 1732398536, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294021, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:48:59.470209 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst) /dev/shm/ToplingTemp/Topling-283069-mGlZZw 2024/11/24-05:49:00.457525 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst): first pass time = 0.99's, 135.948'MB/sec 2024/11/24-05:49:01.774777 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst): old prealloc_size = 147639500, real_size = 31653872 2024/11/24-05:49:01.775375 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919815 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.92 avg-zval = 32.25 2024/11/24-05:49:01.776142 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:02.593070 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294022.sst): second pass time = 1.32's, 82.417'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.670'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3039.776'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 280.228'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33123.794'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.417'MB/sec zip pipeline throughput = 82.417'MB/sec entries = 919815 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.92 avg-zval = 32.25 usrkeys = 919815 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.92 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919815 bytes = 3219360 } seq expand size = 6438705 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4460 value = 3.6566 dict = 1.00 all = 3.7560 } Zip/UnZip{ index = 0.0957 value = 0.2735 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 2252.129258 GB avg = 0.200 KB (by entry num) total key len = 307.064082 GB avg = 0.027 KB total ukey len = 237.484098 GB avg = 0.015 KB total ukey num = 15.745129717 Billion total entry num = 11.286443665 Billion write speed all = 34.018084106 MB/sec (with seq num) write speed all = 32.817883453 MB/sec (without seq num) 2024/11/24-05:49:02.638499 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294022: 919815 keys, 35733778 bytes, temperature: kWarm 2024/11/24-05:49:02.638572 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398542638535, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294022, "file_size": 35733778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4466453046, "table_properties": {"data_size": 29662624, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25754820, "raw_average_key_size": 28, "raw_value_size": 108462994, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919815, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398539, "oldest_key_time": 0, "file_creation_time": 1732398539, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294022, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:02.674528 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst) /dev/shm/ToplingTemp/Topling-283069-5tzRxy 2024/11/24-05:49:03.700095 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst): first pass time = 1.03's, 130.877'MB/sec 2024/11/24-05:49:05.055940 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst): old prealloc_size = 147639500, real_size = 33078000 2024/11/24-05:49:05.056515 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927508 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.71 avg-zval = 33.51 2024/11/24-05:49:05.057300 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:05.909185 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294023.sst): second pass time = 1.35's, 79.914'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.575'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.092'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 290.659'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35081.134'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.914'MB/sec zip pipeline throughput = 79.914'MB/sec entries = 927508 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 33.51 usrkeys = 927508 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927508 bytes = 3246288 } seq expand size = 6492556 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5120 value = 3.4827 dict = 1.00 all = 3.6101 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2252.243998 GB avg = 0.200 KB (by entry num) total key len = 307.090052 GB avg = 0.027 KB total ukey len = 237.502649 GB avg = 0.015 KB total ukey num = 15.746057225 Billion total entry num = 11.287371173 Billion write speed all = 34.018454982 MB/sec (with seq num) write speed all = 32.818208603 MB/sec (without seq num) 2024/11/24-05:49:05.955219 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294023: 927508 keys, 37178790 bytes, temperature: kWarm 2024/11/24-05:49:05.955290 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398545955255, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294023, "file_size": 37178790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4466454122, "table_properties": {"data_size": 31081248, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970224, "raw_average_key_size": 28, "raw_value_size": 108247532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398542, "oldest_key_time": 0, "file_creation_time": 1732398542, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294023, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:05.996204 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst) /dev/shm/ToplingTemp/Topling-283069-iqufRw 2024/11/24-05:49:06.873169 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst): first pass time = 0.88's, 153.056'MB/sec 2024/11/24-05:49:08.194724 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst): old prealloc_size = 147639500, real_size = 31378176 2024/11/24-05:49:08.195285 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.97 2024/11/24-05:49:08.197056 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:09.011952 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294024.sst): second pass time = 1.32's, 82.135'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.386'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3271.303'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 274.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32858.181'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.135'MB/sec zip pipeline throughput = 82.135'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.97 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6912 dict = 1.00 all = 3.7856 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2252.358911 GB avg = 0.200 KB (by entry num) total key len = 307.115793 GB avg = 0.027 KB total ukey len = 237.521035 GB avg = 0.015 KB total ukey num = 15.746976525 Billion total entry num = 11.288290473 Billion write speed all = 34.018921528 MB/sec (with seq num) write speed all = 32.818626898 MB/sec (without seq num) 2024/11/24-05:49:09.055119 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294024: 919300 keys, 35454898 bytes, temperature: kWarm 2024/11/24-05:49:09.055191 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398549055154, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294024, "file_size": 35454898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4314574269, "largest_seqno": 4465509722, "table_properties": {"data_size": 29388016, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398545, "oldest_key_time": 0, "file_creation_time": 1732398545, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294024, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:09.093638 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst) /dev/shm/ToplingTemp/Topling-283069-RFrKLx 2024/11/24-05:49:10.003593 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst): first pass time = 0.91's, 147.506'MB/sec 2024/11/24-05:49:11.334489 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst): old prealloc_size = 147639500, real_size = 33081760 2024/11/24-05:49:11.335006 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927642 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.69 avg-zval = 33.51 2024/11/24-05:49:11.335811 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:12.195265 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294025.sst): second pass time = 1.33's, 81.414'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.198'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3563.474'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 291.739'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32042.786'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.414'MB/sec zip pipeline throughput = 81.414'MB/sec entries = 927642 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.51 usrkeys = 927642 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927642 bytes = 3246768 } seq expand size = 6493494 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5123 value = 3.4822 dict = 1.00 all = 3.6097 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2252.473648 GB avg = 0.200 KB (by entry num) total key len = 307.141767 GB avg = 0.027 KB total ukey len = 237.539587 GB avg = 0.015 KB total ukey num = 15.747904167 Billion total entry num = 11.289218115 Billion write speed all = 34.019352397 MB/sec (with seq num) write speed all = 32.819009917 MB/sec (without seq num) 2024/11/24-05:49:12.240625 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294025: 927642 keys, 37182354 bytes, temperature: kWarm 2024/11/24-05:49:12.240696 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398552240661, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294025, "file_size": 37182354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4466453748, "table_properties": {"data_size": 31084816, "index_size": 1764864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973976, "raw_average_key_size": 28, "raw_value_size": 108243820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398549, "oldest_key_time": 0, "file_creation_time": 1732398549, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294025, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:12.278615 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst) /dev/shm/ToplingTemp/Topling-283069-pY7kIy 2024/11/24-05:49:13.285836 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst): first pass time = 1.01's, 133.261'MB/sec 2024/11/24-05:49:14.606517 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst): old prealloc_size = 147639500, real_size = 31237456 2024/11/24-05:49:14.607088 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927313 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 116.74 avg-zval = 31.53 2024/11/24-05:49:14.607879 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:15.442568 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294026.sst): second pass time = 1.32's, 82.049'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.269'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.653'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 270.558'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27479.564'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.049'MB/sec zip pipeline throughput = 82.049'MB/sec entries = 927313 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.74 avg-zval = 31.53 usrkeys = 927313 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.74 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927313 bytes = 3245616 } seq expand size = 6491191 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5155 value = 3.7020 dict = 1.00 all = 3.7982 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2252.588392 GB avg = 0.200 KB (by entry num) total key len = 307.167731 GB avg = 0.027 KB total ukey len = 237.558134 GB avg = 0.015 KB total ukey num = 15.748831480 Billion total entry num = 11.290145428 Billion write speed all = 34.019754266 MB/sec (with seq num) write speed all = 32.819364996 MB/sec (without seq num) 2024/11/24-05:49:15.489479 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294026: 927313 keys, 35337454 bytes, temperature: kWarm 2024/11/24-05:49:15.489550 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398555489515, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294026, "file_size": 35337454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4466453761, "table_properties": {"data_size": 29241728, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964764, "raw_average_key_size": 28, "raw_value_size": 108253098, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927313, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398552, "oldest_key_time": 0, "file_creation_time": 1732398552, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294026, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:15.531756 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst) /dev/shm/ToplingTemp/Topling-283069-0reBRz 2024/11/24-05:49:16.444731 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst): first pass time = 0.91's, 147.017'MB/sec 2024/11/24-05:49:17.764009 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst): old prealloc_size = 147639500, real_size = 31494976 2024/11/24-05:49:17.764582 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.09 2024/11/24-05:49:17.765377 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:18.579471 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294027.sst): second pass time = 1.32's, 82.305'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.622'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3203.803'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 282.690'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32411.578'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.305'MB/sec zip pipeline throughput = 82.305'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.09 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4455 value = 3.6766 dict = 1.00 all = 3.7732 } Zip/UnZip{ index = 0.0957 value = 0.2720 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 2252.703305 GB avg = 0.200 KB (by entry num) total key len = 307.193472 GB avg = 0.027 KB total ukey len = 237.576520 GB avg = 0.015 KB total ukey num = 15.749750780 Billion total entry num = 11.291064728 Billion write speed all = 34.020205267 MB/sec (with seq num) write speed all = 32.819768302 MB/sec (without seq num) 2024/11/24-05:49:18.628561 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294027: 919300 keys, 35571698 bytes, temperature: kWarm 2024/11/24-05:49:18.628632 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398558628596, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294027, "file_size": 35571698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4272557505, "largest_seqno": 4425081537, "table_properties": {"data_size": 29504752, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398555, "oldest_key_time": 0, "file_creation_time": 1732398555, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294027, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:18.680692 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst) /dev/shm/ToplingTemp/Topling-283069-2gRS3v 2024/11/24-05:49:19.710910 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst): first pass time = 1.03's, 130.286'MB/sec 2024/11/24-05:49:21.039084 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst): old prealloc_size = 147639500, real_size = 33505904 2024/11/24-05:49:21.039630 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927750 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.67 avg-zval = 33.95 2024/11/24-05:49:21.040401 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:21.892028 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294028.sst): second pass time = 1.33's, 81.587'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3390.246'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.582'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32881.191'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.587'MB/sec zip pipeline throughput = 81.587'MB/sec entries = 927750 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.67 avg-zval = 33.95 usrkeys = 927750 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.67 avg-zval = 33.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927750 bytes = 3247136 } seq expand size = 6494250 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4454 value = 3.4365 dict = 1.00 all = 3.5686 } Zip/UnZip{ index = 0.0957 value = 0.2910 dict = 1.00 all = 0.2802 } ---------------------------- total value len = 2252.818040 GB avg = 0.200 KB (by entry num) total key len = 307.219449 GB avg = 0.027 KB total ukey len = 237.595075 GB avg = 0.015 KB total ukey num = 15.750678530 Billion total entry num = 11.291992478 Billion write speed all = 34.020577608 MB/sec (with seq num) write speed all = 32.820094855 MB/sec (without seq num) 2024/11/24-05:49:21.940335 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294028: 927750 keys, 37610692 bytes, temperature: kWarm 2024/11/24-05:49:21.940406 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398561940370, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294028, "file_size": 37610692, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4466453771, "table_properties": {"data_size": 31497376, "index_size": 1776384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977000, "raw_average_key_size": 28, "raw_value_size": 108240740, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398558, "oldest_key_time": 0, "file_creation_time": 1732398558, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294028, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:21.977928 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst) /dev/shm/ToplingTemp/Topling-283069-RAemWx 2024/11/24-05:49:22.859681 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst): first pass time = 0.88's, 152.224'MB/sec 2024/11/24-05:49:24.187535 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst): old prealloc_size = 147639500, real_size = 29765376 2024/11/24-05:49:24.188102 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934718 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0277 GB avg-val = 115.59 avg-zval = 29.68 2024/11/24-05:49:24.188879 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:25.011359 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294029.sst): second pass time = 1.33's, 81.418'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.212'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.12's, 273.058'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32594.704'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.418'MB/sec zip pipeline throughput = 81.418'MB/sec entries = 934718 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.59 avg-zval = 29.68 usrkeys = 934718 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.59 avg-zval = 29.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934718 bytes = 3271520 } seq expand size = 6543026 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0277 GB dict = 1.08 MB all = 0.0339 GB } UnZip/Zip{ index = 10.4448 value = 3.8947 dict = 1.00 all = 3.9609 } Zip/UnZip{ index = 0.0957 value = 0.2568 dict = 1.00 all = 0.2525 } ---------------------------- total value len = 2252.932629 GB avg = 0.199 KB (by entry num) total key len = 307.245621 GB avg = 0.027 KB total ukey len = 237.613769 GB avg = 0.015 KB total ukey num = 15.751613248 Billion total entry num = 11.292927196 Billion write speed all = 34.021037912 MB/sec (with seq num) write speed all = 32.820505553 MB/sec (without seq num) 2024/11/24-05:49:25.055484 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294029: 934718 keys, 33885482 bytes, temperature: kWarm 2024/11/24-05:49:25.055555 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398565055519, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294029, "file_size": 33885482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4466454326, "table_properties": {"data_size": 27741680, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26172104, "raw_average_key_size": 28, "raw_value_size": 108045748, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398561, "oldest_key_time": 0, "file_creation_time": 1732398561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294029, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:25.091631 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst) /dev/shm/ToplingTemp/Topling-283069-50QITx 2024/11/24-05:49:26.051916 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst): first pass time = 0.96's, 139.775'MB/sec 2024/11/24-05:49:27.308516 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst): old prealloc_size = 147639500, real_size = 27738832 2024/11/24-05:49:27.309092 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.18 2024/11/24-05:49:27.309854 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:28.144097 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294030.sst): second pass time = 1.26's, 85.802'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.023'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.754'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 252.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 34968.368'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.802'MB/sec zip pipeline throughput = 85.802'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.18 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4457 value = 4.1939 dict = 1.00 all = 4.2247 } Zip/UnZip{ index = 0.0957 value = 0.2384 dict = 1.00 all = 0.2367 } ---------------------------- total value len = 2253.046997 GB avg = 0.199 KB (by entry num) total key len = 307.272086 GB avg = 0.027 KB total ukey len = 237.632673 GB avg = 0.015 KB total ukey num = 15.752558444 Billion total entry num = 11.293872392 Billion write speed all = 34.021493093 MB/sec (with seq num) write speed all = 32.820910232 MB/sec (without seq num) 2024/11/24-05:49:28.186579 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294030: 945196 keys, 31770032 bytes, temperature: kWarm 2024/11/24-05:49:28.186648 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398568186610, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294030, "file_size": 31770032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4324639462, "largest_seqno": 4426792004, "table_properties": {"data_size": 25692608, "index_size": 1809728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398565, "oldest_key_time": 0, "file_creation_time": 1732398565, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294030, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:28.224026 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst) /dev/shm/ToplingTemp/Topling-283069-C6NXSx 2024/11/24-05:49:29.226165 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst): first pass time = 1.00's, 133.936'MB/sec 2024/11/24-05:49:30.585719 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst): old prealloc_size = 147639500, real_size = 33089984 2024/11/24-05:49:30.586251 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929071 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.46 avg-zval = 33.46 2024/11/24-05:49:30.587060 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:31.437514 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294031.sst): second pass time = 1.36's, 79.656'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.667'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3454.349'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 301.808'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35702.650'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.656'MB/sec zip pipeline throughput = 79.656'MB/sec entries = 929071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.46 avg-zval = 33.46 usrkeys = 929071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.46 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929071 bytes = 3251760 } seq expand size = 6503497 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5110 value = 3.4804 dict = 1.00 all = 3.6085 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 2253.161705 GB avg = 0.199 KB (by entry num) total key len = 307.298100 GB avg = 0.027 KB total ukey len = 237.651254 GB avg = 0.015 KB total ukey num = 15.753487515 Billion total entry num = 11.294801463 Billion write speed all = 34.021874100 MB/sec (with seq num) write speed all = 32.821245018 MB/sec (without seq num) 2024/11/24-05:49:31.484835 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294031: 929071 keys, 37194464 bytes, temperature: kWarm 2024/11/24-05:49:31.484901 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398571484868, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294031, "file_size": 37194464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4466454640, "table_properties": {"data_size": 31089712, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013988, "raw_average_key_size": 28, "raw_value_size": 108203842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398568, "oldest_key_time": 0, "file_creation_time": 1732398568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294031, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:31.520993 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst) /dev/shm/ToplingTemp/Topling-283069-cu2itz 2024/11/24-05:49:32.508079 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst): first pass time = 0.99's, 135.979'MB/sec 2024/11/24-05:49:33.812083 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst): old prealloc_size = 147639500, real_size = 30869840 2024/11/24-05:49:33.812626 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925662 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.00 avg-zval = 31.18 2024/11/24-05:49:33.813406 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:34.634488 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294032.sst): second pass time = 1.30's, 83.130'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3426.682'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 277.453'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28849.454'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.130'MB/sec zip pipeline throughput = 83.130'MB/sec entries = 925662 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.00 avg-zval = 31.18 usrkeys = 925662 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.00 avg-zval = 31.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925662 bytes = 3124128 } seq expand size = 6479634 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4444 value = 3.7518 dict = 1.00 all = 3.8516 } Zip/UnZip{ index = 0.0957 value = 0.2665 dict = 1.00 all = 0.2596 } ---------------------------- total value len = 2253.276484 GB avg = 0.199 KB (by entry num) total key len = 307.324019 GB avg = 0.027 KB total ukey len = 237.669768 GB avg = 0.015 KB total ukey num = 15.754413177 Billion total entry num = 11.295727125 Billion write speed all = 34.022298357 MB/sec (with seq num) write speed all = 32.821621881 MB/sec (without seq num) 2024/11/24-05:49:34.679819 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294032: 925662 keys, 34847158 bytes, temperature: kWarm 2024/11/24-05:49:34.679890 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398574679854, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294032, "file_size": 34847158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4341504362, "largest_seqno": 4466454336, "table_properties": {"data_size": 28865728, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918536, "raw_average_key_size": 28, "raw_value_size": 108299324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925662, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398571, "oldest_key_time": 0, "file_creation_time": 1732398571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294032, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:34.727167 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst) /dev/shm/ToplingTemp/Topling-283069-ptfyux 2024/11/24-05:49:35.708154 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst): first pass time = 0.98's, 136.826'MB/sec 2024/11/24-05:49:36.982147 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst): old prealloc_size = 147639500, real_size = 28363760 2024/11/24-05:49:36.982693 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944374 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0263 GB avg-val = 114.12 avg-zval = 27.88 2024/11/24-05:49:36.984701 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:37.817692 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294033.sst): second pass time = 1.27's, 84.647'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.913'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3429.086'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 257.503'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 34112.499'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.647'MB/sec zip pipeline throughput = 84.647'MB/sec entries = 944374 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.12 avg-zval = 27.88 usrkeys = 944374 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.12 avg-zval = 27.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944374 bytes = 3305328 } seq expand size = 6610618 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0263 GB dict = 1.07 MB all = 0.0325 GB } UnZip/Zip{ index = 10.5129 value = 4.0931 dict = 1.00 all = 4.1282 } Zip/UnZip{ index = 0.0951 value = 0.2443 dict = 1.00 all = 0.2422 } ---------------------------- total value len = 2253.390870 GB avg = 0.199 KB (by entry num) total key len = 307.350461 GB avg = 0.027 KB total ukey len = 237.688655 GB avg = 0.015 KB total ukey num = 15.755357551 Billion total entry num = 11.296671499 Billion write speed all = 34.022730538 MB/sec (with seq num) write speed all = 32.822004465 MB/sec (without seq num) 2024/11/24-05:49:37.861352 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294033: 944374 keys, 32512282 bytes, temperature: kWarm 2024/11/24-05:49:37.861422 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398577861386, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294033, "file_size": 32512282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4466453080, "table_properties": {"data_size": 26330848, "index_size": 1796608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26442472, "raw_average_key_size": 28, "raw_value_size": 107775376, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944374, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398574, "oldest_key_time": 0, "file_creation_time": 1732398574, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294033, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:37.898902 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst) /dev/shm/ToplingTemp/Topling-283069-STHtYz 2024/11/24-05:49:38.893057 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst): first pass time = 0.99's, 135.012'MB/sec 2024/11/24-05:49:40.197112 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst): old prealloc_size = 147639500, real_size = 31340976 2024/11/24-05:49:40.197679 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-05:49:40.198435 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:41.005167 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294034.sst): second pass time = 1.30's, 83.237'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.690'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.243'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 282.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 29809.605'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.237'MB/sec zip pipeline throughput = 83.237'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4462 value = 3.6959 dict = 1.00 all = 3.8019 } Zip/UnZip{ index = 0.0957 value = 0.2706 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2253.505782 GB avg = 0.199 KB (by entry num) total key len = 307.376202 GB avg = 0.027 KB total ukey len = 237.707041 GB avg = 0.015 KB total ukey num = 15.756276851 Billion total entry num = 11.297590799 Billion write speed all = 34.023158422 MB/sec (with seq num) write speed all = 32.822385489 MB/sec (without seq num) 2024/11/24-05:49:41.048124 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294034: 919300 keys, 35302834 bytes, temperature: kWarm 2024/11/24-05:49:41.048196 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398581048160, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294034, "file_size": 35302834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269074288, "largest_seqno": 4385090671, "table_properties": {"data_size": 29350880, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398577, "oldest_key_time": 0, "file_creation_time": 1732398577, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294034, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:41.086019 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst) /dev/shm/ToplingTemp/Topling-283069-GEd9Kw 2024/11/24-05:49:41.950262 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst): first pass time = 0.86's, 155.308'MB/sec 2024/11/24-05:49:43.252300 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst): old prealloc_size = 147639500, real_size = 30561600 2024/11/24-05:49:43.252858 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927848 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.66 avg-zval = 30.77 2024/11/24-05:49:43.253652 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:44.064077 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294035.sst): second pass time = 1.30's, 83.174'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.557'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3425.303'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 274.932'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35743.597'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.174'MB/sec zip pipeline throughput = 83.174'MB/sec entries = 927848 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.66 avg-zval = 30.77 usrkeys = 927848 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.66 avg-zval = 30.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927848 bytes = 3131504 } seq expand size = 6494936 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4412 value = 3.7909 dict = 1.00 all = 3.8853 } Zip/UnZip{ index = 0.0958 value = 0.2638 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 2253.620515 GB avg = 0.199 KB (by entry num) total key len = 307.402181 GB avg = 0.027 KB total ukey len = 237.725598 GB avg = 0.015 KB total ukey num = 15.757204699 Billion total entry num = 11.298518647 Billion write speed all = 34.023645179 MB/sec (with seq num) write speed all = 32.822822429 MB/sec (without seq num) 2024/11/24-05:49:44.108125 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294035: 927848 keys, 34545204 bytes, temperature: kWarm 2024/11/24-05:49:44.108198 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398584108163, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294035, "file_size": 34545204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4385090672, "largest_seqno": 4466454349, "table_properties": {"data_size": 28552176, "index_size": 1777280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979744, "raw_average_key_size": 28, "raw_value_size": 108238112, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927848, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398581, "oldest_key_time": 0, "file_creation_time": 1732398581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294035, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:44.147814 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst) /dev/shm/ToplingTemp/Topling-283069-BNeHNx 2024/11/24-05:49:45.060919 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst): first pass time = 0.91's, 146.996'MB/sec 2024/11/24-05:49:46.410016 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst): old prealloc_size = 147639500, real_size = 32987744 2024/11/24-05:49:46.410585 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928410 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.57 avg-zval = 33.38 2024/11/24-05:49:46.412076 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:47.264239 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294036.sst): second pass time = 1.35's, 80.300'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3232.188'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 296.809'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30779.977'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.300'MB/sec zip pipeline throughput = 80.300'MB/sec entries = 928410 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.38 usrkeys = 928410 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928410 bytes = 3249456 } seq expand size = 6498870 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5142 value = 3.4922 dict = 1.00 all = 3.6187 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 2253.735236 GB avg = 0.199 KB (by entry num) total key len = 307.428177 GB avg = 0.027 KB total ukey len = 237.744166 GB avg = 0.015 KB total ukey num = 15.758133109 Billion total entry num = 11.299447057 Billion write speed all = 34.024068102 MB/sec (with seq num) write speed all = 32.823197734 MB/sec (without seq num) 2024/11/24-05:49:47.310948 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294036: 928410 keys, 37090042 bytes, temperature: kWarm 2024/11/24-05:49:47.311034 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398587310996, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294036, "file_size": 37090042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4466453609, "table_properties": {"data_size": 30989456, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995480, "raw_average_key_size": 28, "raw_value_size": 108222376, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928410, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398584, "oldest_key_time": 0, "file_creation_time": 1732398584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294036, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:47.354494 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst) /dev/shm/ToplingTemp/Topling-283069-nDenwz 2024/11/24-05:49:48.255339 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst): first pass time = 0.90's, 148.997'MB/sec 2024/11/24-05:49:49.573404 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst): old prealloc_size = 147639500, real_size = 31300256 2024/11/24-05:49:49.573998 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.88 2024/11/24-05:49:49.574737 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:50.384219 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294037.sst): second pass time = 1.32's, 82.346'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.491'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3087.641'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 280.069'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36622.338'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.346'MB/sec zip pipeline throughput = 82.346'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.88 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4458 value = 3.7010 dict = 1.00 all = 3.8063 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2253.850149 GB avg = 0.199 KB (by entry num) total key len = 307.453917 GB avg = 0.027 KB total ukey len = 237.762552 GB avg = 0.015 KB total ukey num = 15.759052409 Billion total entry num = 11.300366357 Billion write speed all = 34.024526381 MB/sec (with seq num) write speed all = 32.823608089 MB/sec (without seq num) 2024/11/24-05:49:50.429666 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294037: 919300 keys, 35262066 bytes, temperature: kWarm 2024/11/24-05:49:50.429734 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398590429699, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294037, "file_size": 35262066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4335455086, "largest_seqno": 4447852957, "table_properties": {"data_size": 29310096, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398587, "oldest_key_time": 0, "file_creation_time": 1732398587, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294037, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:50.470541 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst) /dev/shm/ToplingTemp/Topling-283069-PweVzw 2024/11/24-05:49:51.487101 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst): first pass time = 1.02's, 132.037'MB/sec 2024/11/24-05:49:52.819633 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst): old prealloc_size = 147639500, real_size = 32329968 2024/11/24-05:49:52.820217 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932670 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.91 avg-zval = 32.51 2024/11/24-05:49:52.820997 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:49:53.673252 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294038.sst): second pass time = 1.33's, 81.173'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.696'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.697'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 290.226'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34075.739'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.173'MB/sec zip pipeline throughput = 81.173'MB/sec entries = 932670 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.91 avg-zval = 32.51 usrkeys = 932670 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.91 avg-zval = 32.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932670 bytes = 3264352 } seq expand size = 6528690 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5114 value = 3.5652 dict = 1.00 all = 3.6828 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 2253.964781 GB avg = 0.199 KB (by entry num) total key len = 307.480032 GB avg = 0.027 KB total ukey len = 237.781206 GB avg = 0.015 KB total ukey num = 15.759985079 Billion total entry num = 11.301299027 Billion write speed all = 34.024909460 MB/sec (with seq num) write speed all = 32.823944522 MB/sec (without seq num) 2024/11/24-05:49:53.718933 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294038: 932670 keys, 36444978 bytes, temperature: kWarm 2024/11/24-05:49:53.719020 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398593718968, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294038, "file_size": 36444978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4466454367, "table_properties": {"data_size": 30322016, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26114760, "raw_average_key_size": 28, "raw_value_size": 108103000, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398590, "oldest_key_time": 0, "file_creation_time": 1732398590, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:53.755300 139957441766976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst) /dev/shm/ToplingTemp/Topling-283069-PDiT0y 2024/11/24-05:49:53.835808 139957441766976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst): first pass time = 0.08's, 239.681'MB/sec 2024/11/24-05:49:54.018174 139957441766976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst): old prealloc_size = 147639500, real_size = 5698672 2024/11/24-05:49:54.018320 139957441766976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 132093 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0156 GB zip-val = 0.0054 GB avg-val = 118.00 avg-zval = 40.97 2024/11/24-05:49:54.018454 139957441766976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/24-05:49:54.136952 139957441766976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00/294039.sst): second pass time = 0.18's, 85.795'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 52.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2048.557'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.332'MB/sec write SST data time = 0.02's, 326.489'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.162'MB, throughput = 3207.713'MB/sec zip my value time = 0.18's, unzip length = 0.016'GB zip my value throughput = 85.795'MB/sec zip pipeline throughput = 85.795'MB/sec entries = 132093 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 40.97 usrkeys = 132093 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 40.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 132093 bytes = 396288 } seq expand size = 924651 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0026 GB value = 0.0156 GB dict = 0.16 MB all = 0.0193 GB } __ZipSize{ index = 0.0003 GB value = 0.0054 GB dict = 0.16 MB all = 0.0062 GB } UnZip/Zip{ index = 10.3951 value = 2.8805 dict = 1.00 all = 3.0977 } Zip/UnZip{ index = 0.0962 value = 0.3472 dict = 1.00 all = 0.3228 } ---------------------------- total value len = 2253.981292 GB avg = 0.199 KB (by entry num) total key len = 307.483731 GB avg = 0.027 KB total ukey len = 237.783848 GB avg = 0.015 KB total ukey num = 15.760117172 Billion total entry num = 11.301431120 Billion write speed all = 34.024968341 MB/sec (with seq num) write speed all = 32.823996764 MB/sec (without seq num) 2024/11/24-05:49:54.150682 139957441766976 [compaction_job.cc:1948] [default] [JOB 42409] Generated table #294039: 132093 keys, 6225823 bytes, temperature: kWarm 2024/11/24-05:49:54.150737 139957441766976 EVENT_LOG_v1 {"time_micros": 1732398594150708, "cf_name": "default", "job": 42409, "event": "table_file_creation", "file_number": 294039, "file_size": 6225823, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4299386578, "largest_seqno": 4314918150, "table_properties": {"data_size": 5411296, "index_size": 254144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3698604, "raw_average_key_size": 28, "raw_value_size": 15586974, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 132093, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732398593, "oldest_key_time": 0, "file_creation_time": 1732398593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:49:54.160757 139957441766976 [compaction_job.cc:703] job-42409: subcompact[0], size: 0.824083 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42409/att-00 [294016,294017,294018,294019,294020,294021,294022,294023,294024,294025,294026,294027,294028,294029,294030,294031,294032,294033,294034,294035,294036,294037,294038,294039] 2024/11/24-05:49:54.298194 139957441766976 (Original Log Time 2024/11/24-05:48:36.807545) EVENT_LOG_v1 {"time_micros": 1732398516807485, "job": 42409, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294011], "files_L3": [293948, 293949, 293950, 293951, 293952, 293953, 293954, 293957, 293958, 293959, 293960, 293961, 293962, 293963, 293964, 293965, 293966, 293967, 293968, 293969, 293970, 293971, 293972], "score": 0.781138, "input_data_size": 848278492, "oldest_snapshot_seqno": 4466480330} 2024/11/24-05:49:54.298198 139957441766976 (Original Log Time 2024/11/24-05:49:54.296608) [compaction_job.cc:2022] [default] [JOB 42409] Compacted 1@2 + 23@3 files to L3 => 824083467 bytes 2024/11/24-05:49:54.298200 139957441766976 (Original Log Time 2024/11/24-05:49:54.298121) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 23) out(24 +0 blob) MB in(27.1, 781.9 +0.0 blob) out(785.9 +0.0 blob), read-write-amplify(58.9) write-amplify(29.0) OK, records in: 21556283, records dropped: 100274 output_compression: Snappy 2024/11/24-05:49:54.298203 139957441766976 (Original Log Time 2024/11/24-05:49:54.298167) EVENT_LOG_v1 {"time_micros": 1732398594298145, "cf": "default", "job": 42409, "event": "compaction_finished", "compaction_time_micros": 77353217, "compaction_time_cpu_micros": 33170466, "output_level": 3, "num_output_files": 24, "total_output_size": 824083467, "num_input_records": 21556283, "num_output_records": 21456009, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 599846698, "file_range_sync_nanos": 0, "file_fsync_nanos": 16687, "file_prepare_write_nanos": 67788, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/24-05:49:54.439878 139957441766976 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732398516,"executesMs":77641,"compactionJobId":42409,"attempt":0,"compactionInputRawBytes":3120534118,"compactionInputZipBytes":848278492,"compactionOutputRawBytes":0,"compactionOutputZipBytes":824083467,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-93632e5a0097394be2711af9e6b2da32-f7975fbc33baf071-00"}}, response = 2024/11/24-05:49:54.439954 139957441766976 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42409/att-00: olev 3, work 77.641 s, result 7.736 ms, install 1.611 ms, input{raw 2.906 GiB zip 808.981 MiB}