2024/11/24-03:52:16.955695 139957676762688 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00: fork to child time = 0.021641 sec 2024/11/24-03:52:16.956572 139957676762688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00 2024/11/24-03:52:16.963779 139957676762688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00 2024/11/24-03:52:16.964996 139957676762688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:52:16.965374 139957676762688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:52:17.015827 139957676762688 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41097/att-00/MANIFEST-206347 2024/11/24-03:52:17.042107 139957676762688 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41097/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290988, last_sequence is 4426433166, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290816 2024/11/24-03:52:17.042121 139957676762688 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:52:17.042329 139957676762688 [version_set.cc:5721] Creating manifest 290988 2024/11/24-03:52:17.077693 139957676762688 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:52:17.097941 139957676762688 [compaction_job.cc:2437] [default] [JOB 41097] Compacting 1@2 + 21@3 files to L3, score 0.64, subcompactions 1 : 1 2024/11/24-03:52:17.097958 139957676762688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30778 Base level 2, inputs: [290947(27MB)], [290977(35MB) 290978(1688KB) 290922(35MB) 290923(34MB) 290924(3474KB) 290890(33MB) 290891(34MB) 290892(35MB) 290893(30MB) 290894(34MB) 290895(33MB) 290896(32MB) 290897(33MB) 290898(33MB) 290899(33MB) 290900(34MB) 290901(33MB) 290902(35MB) 290903(33MB) 290904(35MB) 290979(33MB)] 2024/11/24-03:52:17.105223 139957676762688 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.105253 139957676762688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.105271 139957676762688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.105277 139957676762688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:52:17.105288 139957676762688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:52:17.679073 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst) /dev/shm/ToplingTemp/Topling-262866-KkFE3S 2024/11/24-03:52:18.247702 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst): first pass time = 0.57's, 236.055'MB/sec 2024/11/24-03:52:20.157349 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst): old prealloc_size = 147639500, real_size = 32969152 2024/11/24-03:52:20.157910 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924258 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.22 avg-zval = 33.52 2024/11/24-03:52:20.158713 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:21.315342 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290989.sst): second pass time = 1.91's, 56.761'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 32.004'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.809'MB/sec (index lex order gen) rebuild zvType time = 1.02's, 0.227'MB/sec write SST data time = 0.14's, 265.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36890.784'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.761'MB/sec zip pipeline throughput = 86.244'MB/sec entries = 924258 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 33.52 usrkeys = 924258 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924258 bytes = 3234912 } seq expand size = 6469806 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5144 value = 3.4971 dict = 1.00 all = 3.6216 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 2251.440382 GB avg = 0.200 KB (by entry num) total key len = 306.908842 GB avg = 0.027 KB total ukey len = 237.373212 GB avg = 0.015 KB total ukey num = 15.739585421 Billion total entry num = 11.280899369 Billion write speed all = 37.496452351 MB/sec (with seq num) write speed all = 36.173744093 MB/sec (without seq num) 2024/11/24-03:52:21.362708 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290989: 924258 keys, 37060396 bytes, temperature: kWarm 2024/11/24-03:52:21.362798 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391541362744, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290989, "file_size": 37060396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4426236135, "table_properties": {"data_size": 30979824, "index_size": 1758080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25879224, "raw_average_key_size": 28, "raw_value_size": 108338588, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391537, "oldest_key_time": 0, "file_creation_time": 1732391537, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290989, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:21.397178 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst) /dev/shm/ToplingTemp/Topling-262866-tvtlfS 2024/11/24-03:52:22.277585 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst): first pass time = 0.88's, 152.458'MB/sec 2024/11/24-03:52:23.810819 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst): old prealloc_size = 147639500, real_size = 33406048 2024/11/24-03:52:23.811379 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926769 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.82 avg-zval = 33.89 2024/11/24-03:52:23.812150 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:24.668806 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290990.sst): second pass time = 1.53's, 70.650'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.713'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.856'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 297.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33550.352'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.650'MB/sec zip pipeline throughput = 87.347'MB/sec entries = 926769 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 33.89 usrkeys = 926769 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 33.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926769 bytes = 3243712 } seq expand size = 6487383 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5127 value = 3.4468 dict = 1.00 all = 3.5787 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 2251.555137 GB avg = 0.200 KB (by entry num) total key len = 306.934791 GB avg = 0.027 KB total ukey len = 237.391748 GB avg = 0.015 KB total ukey num = 15.740512190 Billion total entry num = 11.281826138 Billion write speed all = 37.496671640 MB/sec (with seq num) write speed all = 36.173919729 MB/sec (without seq num) 2024/11/24-03:52:24.716283 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290990: 926769 keys, 37504310 bytes, temperature: kWarm 2024/11/24-03:52:24.716358 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391544716319, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290990, "file_size": 37504310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4426239128, "table_properties": {"data_size": 31411024, "index_size": 1763136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949532, "raw_average_key_size": 28, "raw_value_size": 108268278, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926769, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391541, "oldest_key_time": 0, "file_creation_time": 1732391541, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290990, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:24.762346 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst) /dev/shm/ToplingTemp/Topling-262866-sfGh0T 2024/11/24-03:52:25.714489 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst): first pass time = 0.95's, 140.970'MB/sec 2024/11/24-03:52:27.154155 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst): old prealloc_size = 147639500, real_size = 31797104 2024/11/24-03:52:27.154716 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926885 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 116.81 avg-zval = 32.15 2024/11/24-03:52:27.155471 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:27.983176 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290991.sst): second pass time = 1.44's, 75.248'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.300'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3274.678'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 283.332'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27702.028'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.248'MB/sec zip pipeline throughput = 75.248'MB/sec entries = 926885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 32.15 usrkeys = 926885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 32.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926885 bytes = 3244112 } seq expand size = 6488195 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5140 value = 3.6328 dict = 1.00 all = 3.7391 } Zip/UnZip{ index = 0.0951 value = 0.2753 dict = 1.00 all = 0.2674 } ---------------------------- total value len = 2251.669891 GB avg = 0.200 KB (by entry num) total key len = 306.960744 GB avg = 0.027 KB total ukey len = 237.410286 GB avg = 0.015 KB total ukey num = 15.741439075 Billion total entry num = 11.282753023 Billion write speed all = 37.496912396 MB/sec (with seq num) write speed all = 36.174116066 MB/sec (without seq num) 2024/11/24-03:52:28.029032 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290991: 926885 keys, 35895672 bytes, temperature: kWarm 2024/11/24-03:52:28.029101 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391548029065, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290991, "file_size": 35895672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4426238354, "table_properties": {"data_size": 29802080, "index_size": 1763136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952780, "raw_average_key_size": 28, "raw_value_size": 108265066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391544, "oldest_key_time": 0, "file_creation_time": 1732391544, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290991, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:28.067224 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst) /dev/shm/ToplingTemp/Topling-262866-qcb9qR 2024/11/24-03:52:29.093535 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst): first pass time = 1.03's, 130.782'MB/sec 2024/11/24-03:52:30.529771 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst): old prealloc_size = 147639500, real_size = 31472544 2024/11/24-03:52:30.530376 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920507 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0295 GB avg-val = 117.81 avg-zval = 32.03 2024/11/24-03:52:30.531183 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:31.353571 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290992.sst): second pass time = 1.43's, 75.574'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.158'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3008.244'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 282.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34138.841'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.574'MB/sec zip pipeline throughput = 75.574'MB/sec entries = 920507 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.81 avg-zval = 32.03 usrkeys = 920507 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.81 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920507 bytes = 3221792 } seq expand size = 6443549 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4451 value = 3.6786 dict = 1.00 all = 3.7752 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.784778 GB avg = 0.200 KB (by entry num) total key len = 306.986518 GB avg = 0.027 KB total ukey len = 237.428696 GB avg = 0.015 KB total ukey num = 15.742359582 Billion total entry num = 11.283673530 Billion write speed all = 37.497121689 MB/sec (with seq num) write speed all = 36.174282778 MB/sec (without seq num) 2024/11/24-03:52:31.398283 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290992: 920507 keys, 35552528 bytes, temperature: kWarm 2024/11/24-03:52:31.398353 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391551398318, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290992, "file_size": 35552528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280964997, "largest_seqno": 4426238372, "table_properties": {"data_size": 29479696, "index_size": 1762560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25774196, "raw_average_key_size": 28, "raw_value_size": 108443534, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920507, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391548, "oldest_key_time": 0, "file_creation_time": 1732391548, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290992, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:31.439853 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst) /dev/shm/ToplingTemp/Topling-262866-Vbj3YU 2024/11/24-03:52:32.458624 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst): first pass time = 1.02's, 131.750'MB/sec 2024/11/24-03:52:33.922659 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst): old prealloc_size = 147639500, real_size = 32978336 2024/11/24-03:52:33.923448 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925347 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.05 avg-zval = 33.49 2024/11/24-03:52:33.924305 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:34.799798 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290993.sst): second pass time = 1.46's, 74.018'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2299.081'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.15's, 251.491'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32459.019'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.018'MB/sec zip pipeline throughput = 74.018'MB/sec entries = 925347 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.49 usrkeys = 925347 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925347 bytes = 3238736 } seq expand size = 6477429 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5145 value = 3.4953 dict = 1.00 all = 3.6205 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 2251.899563 GB avg = 0.200 KB (by entry num) total key len = 307.012428 GB avg = 0.027 KB total ukey len = 237.447203 GB avg = 0.015 KB total ukey num = 15.743284929 Billion total entry num = 11.284598877 Billion write speed all = 37.497289791 MB/sec (with seq num) write speed all = 36.174409205 MB/sec (without seq num) 2024/11/24-03:52:34.845714 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290993: 925347 keys, 37071934 bytes, temperature: kWarm 2024/11/24-03:52:34.845785 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391554845749, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290993, "file_size": 37071934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4426239234, "table_properties": {"data_size": 30986704, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909716, "raw_average_key_size": 28, "raw_value_size": 108308122, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925347, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391551, "oldest_key_time": 0, "file_creation_time": 1732391551, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290993, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:34.884738 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst) /dev/shm/ToplingTemp/Topling-262866-yPoGrV 2024/11/24-03:52:35.881408 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst): first pass time = 1.00's, 134.672'MB/sec 2024/11/24-03:52:37.349498 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst): old prealloc_size = 147639500, real_size = 33605136 2024/11/24-03:52:37.350077 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926932 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.80 avg-zval = 34.09 2024/11/24-03:52:37.350823 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:38.209147 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290994.sst): second pass time = 1.47's, 73.787'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.865'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3194.828'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 296.819'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27119.803'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.787'MB/sec zip pipeline throughput = 73.787'MB/sec entries = 926932 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 34.09 usrkeys = 926932 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926932 bytes = 3244272 } seq expand size = 6488524 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4444 value = 3.4263 dict = 1.00 all = 3.5594 } Zip/UnZip{ index = 0.0957 value = 0.2919 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 2252.014315 GB avg = 0.200 KB (by entry num) total key len = 307.038382 GB avg = 0.027 KB total ukey len = 237.465741 GB avg = 0.015 KB total ukey num = 15.744211861 Billion total entry num = 11.285525809 Billion write speed all = 37.497478300 MB/sec (with seq num) write speed all = 36.174555143 MB/sec (without seq num) 2024/11/24-03:52:38.260027 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290994: 926932 keys, 37707580 bytes, temperature: kWarm 2024/11/24-03:52:38.260099 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391558260063, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290994, "file_size": 37707580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4426238387, "table_properties": {"data_size": 31598272, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954096, "raw_average_key_size": 28, "raw_value_size": 108263716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391554, "oldest_key_time": 0, "file_creation_time": 1732391554, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290994, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:38.303938 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst) /dev/shm/ToplingTemp/Topling-262866-sBUi6T 2024/11/24-03:52:39.135993 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst): first pass time = 0.83's, 161.317'MB/sec 2024/11/24-03:52:40.539590 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst): old prealloc_size = 147639500, real_size = 28895712 2024/11/24-03:52:40.540134 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939134 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1079 GB zip-val = 0.0269 GB avg-val = 114.92 avg-zval = 28.59 2024/11/24-03:52:40.540911 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:41.372592 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290995.sst): second pass time = 1.40's, 76.928'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.652'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.741'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 262.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36719.588'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 76.928'MB/sec zip pipeline throughput = 76.928'MB/sec entries = 939134 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.92 avg-zval = 28.59 usrkeys = 939134 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.92 avg-zval = 28.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939134 bytes = 3286976 } seq expand size = 6573938 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0269 GB dict = 1.08 MB all = 0.0330 GB } UnZip/Zip{ index = 10.3769 value = 4.0193 dict = 1.00 all = 4.0636 } Zip/UnZip{ index = 0.0964 value = 0.2488 dict = 1.00 all = 0.2461 } ---------------------------- total value len = 2252.128811 GB avg = 0.200 KB (by entry num) total key len = 307.064678 GB avg = 0.027 KB total ukey len = 237.484524 GB avg = 0.015 KB total ukey num = 15.745150995 Billion total entry num = 11.286464943 Billion write speed all = 37.497803147 MB/sec (with seq num) write speed all = 36.174831227 MB/sec (without seq num) 2024/11/24-03:52:41.415529 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290995: 939134 keys, 33029396 bytes, temperature: kWarm 2024/11/24-03:52:41.415598 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391561415563, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290995, "file_size": 33029396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4426239526, "table_properties": {"data_size": 26850704, "index_size": 1810048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26295752, "raw_average_key_size": 28, "raw_value_size": 107922052, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939134, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391558, "oldest_key_time": 0, "file_creation_time": 1732391558, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290995, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:41.453884 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst) /dev/shm/ToplingTemp/Topling-262866-bEdjzV 2024/11/24-03:52:42.390292 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst): first pass time = 0.94's, 143.338'MB/sec 2024/11/24-03:52:43.823720 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst): old prealloc_size = 147639500, real_size = 30409264 2024/11/24-03:52:43.824278 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 940636 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1079 GB zip-val = 0.0284 GB avg-val = 114.69 avg-zval = 30.18 2024/11/24-03:52:43.825045 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:44.670011 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290996.sst): second pass time = 1.43's, 75.304'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.825'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.409'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.12's, 277.166'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 35932.478'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.304'MB/sec zip pipeline throughput = 75.304'MB/sec entries = 940636 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.69 avg-zval = 30.18 usrkeys = 940636 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.69 avg-zval = 30.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 940636 bytes = 3292240 } seq expand size = 6584452 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5128 value = 3.8007 dict = 1.00 all = 3.8849 } Zip/UnZip{ index = 0.0951 value = 0.2631 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 2252.243276 GB avg = 0.200 KB (by entry num) total key len = 307.091016 GB avg = 0.027 KB total ukey len = 237.503337 GB avg = 0.015 KB total ukey num = 15.746091631 Billion total entry num = 11.287405579 Billion write speed all = 37.498054519 MB/sec (with seq num) write speed all = 36.175036261 MB/sec (without seq num) 2024/11/24-03:52:44.713077 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290996: 940636 keys, 34548306 bytes, temperature: kWarm 2024/11/24-03:52:44.713145 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391564713110, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290996, "file_size": 34548306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4426239716, "table_properties": {"data_size": 28384416, "index_size": 1789504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26337808, "raw_average_key_size": 28, "raw_value_size": 107879936, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 940636, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391561, "oldest_key_time": 0, "file_creation_time": 1732391561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290996, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:44.750328 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst) /dev/shm/ToplingTemp/Topling-262866-HKwBFT 2024/11/24-03:52:45.640093 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst): first pass time = 0.89's, 150.853'MB/sec 2024/11/24-03:52:47.165467 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst): old prealloc_size = 147639500, real_size = 31268048 2024/11/24-03:52:47.165972 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920845 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.76 avg-zval = 31.79 2024/11/24-03:52:47.166800 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:47.985927 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290997.sst): second pass time = 1.52's, 71.122'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.086'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3490.471'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 273.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36689.725'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.122'MB/sec zip pipeline throughput = 113.028'MB/sec entries = 920845 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.76 avg-zval = 31.79 usrkeys = 920845 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.76 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920845 bytes = 3222976 } seq expand size = 6445915 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4436 value = 3.7041 dict = 1.00 all = 3.7969 } Zip/UnZip{ index = 0.0958 value = 0.2700 dict = 1.00 all = 0.2634 } ---------------------------- total value len = 2252.358156 GB avg = 0.200 KB (by entry num) total key len = 307.116799 GB avg = 0.027 KB total ukey len = 237.521754 GB avg = 0.015 KB total ukey num = 15.747012476 Billion total entry num = 11.288326424 Billion write speed all = 37.498293658 MB/sec (with seq num) write speed all = 36.175231744 MB/sec (without seq num) 2024/11/24-03:52:48.046364 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290997: 920845 keys, 35348892 bytes, temperature: kWarm 2024/11/24-03:52:48.046421 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391568046389, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290997, "file_size": 35348892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4288802966, "largest_seqno": 4426239539, "table_properties": {"data_size": 29274176, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25783660, "raw_average_key_size": 28, "raw_value_size": 108434114, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920845, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391564, "oldest_key_time": 0, "file_creation_time": 1732391564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290997, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:48.092393 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst) /dev/shm/ToplingTemp/Topling-262866-UHk38S 2024/11/24-03:52:48.967602 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst): first pass time = 0.88's, 153.362'MB/sec 2024/11/24-03:52:50.460377 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst): old prealloc_size = 147639500, real_size = 27730400 2024/11/24-03:52:50.460953 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.16 2024/11/24-03:52:50.461720 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:51.303823 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290998.sst): second pass time = 1.49's, 72.243'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.658'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.841'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 242.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 32942.274'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.243'MB/sec zip pipeline throughput = 126.887'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 27.16 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 27.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.3753 value = 4.1973 dict = 1.00 all = 4.2258 } Zip/UnZip{ index = 0.0964 value = 0.2382 dict = 1.00 all = 0.2366 } ---------------------------- total value len = 2252.472525 GB avg = 0.200 KB (by entry num) total key len = 307.143265 GB avg = 0.027 KB total ukey len = 237.540658 GB avg = 0.015 KB total ukey num = 15.747957672 Billion total entry num = 11.289271620 Billion write speed all = 37.498534193 MB/sec (with seq num) write speed all = 36.175425812 MB/sec (without seq num) 2024/11/24-03:52:51.370909 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290998: 945196 keys, 31761584 bytes, temperature: kWarm 2024/11/24-03:52:51.370996 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391571370946, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290998, "file_size": 31761584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4321545299, "largest_seqno": 4423717773, "table_properties": {"data_size": 25671888, "index_size": 1822016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391568, "oldest_key_time": 0, "file_creation_time": 1732391568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290998, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:51.406592 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst) /dev/shm/ToplingTemp/Topling-262866-574iPV 2024/11/24-03:52:52.250389 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst): first pass time = 0.84's, 159.072'MB/sec 2024/11/24-03:52:53.855799 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst): old prealloc_size = 147639500, real_size = 31612016 2024/11/24-03:52:53.856471 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919907 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.90 avg-zval = 32.19 2024/11/24-03:52:53.857303 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:54.684996 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/290999.sst): second pass time = 1.60's, 67.613'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.046'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2728.168'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.14's, 253.171'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32942.717'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.613'MB/sec zip pipeline throughput = 127.075'MB/sec entries = 919907 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.90 avg-zval = 32.19 usrkeys = 919907 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.90 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919907 bytes = 3219696 } seq expand size = 6439349 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.3784 value = 3.6631 dict = 1.00 all = 3.7604 } Zip/UnZip{ index = 0.0964 value = 0.2730 dict = 1.00 all = 0.2659 } ---------------------------- total value len = 2252.587424 GB avg = 0.200 KB (by entry num) total key len = 307.169022 GB avg = 0.027 KB total ukey len = 237.559056 GB avg = 0.015 KB total ukey num = 15.748877579 Billion total entry num = 11.290191527 Billion write speed all = 37.498737361 MB/sec (with seq num) write speed all = 36.175586707 MB/sec (without seq num) 2024/11/24-03:52:54.740057 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #290999: 919907 keys, 35692046 bytes, temperature: kWarm 2024/11/24-03:52:54.740131 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391574740093, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 290999, "file_size": 35692046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4426237985, "table_properties": {"data_size": 29609120, "index_size": 1772736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25757396, "raw_average_key_size": 28, "raw_value_size": 108460426, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919907, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391571, "oldest_key_time": 0, "file_creation_time": 1732391571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290999, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:54.776160 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst) /dev/shm/ToplingTemp/Topling-262866-sTHS2S 2024/11/24-03:52:55.660239 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst): first pass time = 0.88's, 151.823'MB/sec 2024/11/24-03:52:57.207706 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst): old prealloc_size = 147639500, real_size = 29619872 2024/11/24-03:52:57.208315 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934936 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0276 GB avg-val = 115.56 avg-zval = 29.52 2024/11/24-03:52:57.209039 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:58.031557 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291000.sst): second pass time = 1.55's, 69.873'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3050.266'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 262.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33656.212'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.873'MB/sec zip pipeline throughput = 120.792'MB/sec entries = 934936 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.56 avg-zval = 29.52 usrkeys = 934936 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.56 avg-zval = 29.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934936 bytes = 3038560 } seq expand size = 6544552 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0276 GB dict = 1.08 MB all = 0.0335 GB } UnZip/Zip{ index = 10.4416 value = 3.9152 dict = 1.00 all = 4.0057 } Zip/UnZip{ index = 0.0958 value = 0.2554 dict = 1.00 all = 0.2496 } ---------------------------- total value len = 2252.702008 GB avg = 0.200 KB (by entry num) total key len = 307.195200 GB avg = 0.027 KB total ukey len = 237.577754 GB avg = 0.015 KB total ukey num = 15.749812515 Billion total entry num = 11.291126463 Billion write speed all = 37.498961044 MB/sec (with seq num) write speed all = 36.175765691 MB/sec (without seq num) 2024/11/24-03:52:58.073121 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291000: 934936 keys, 33507070 bytes, temperature: kWarm 2024/11/24-03:52:58.073191 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391578073156, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291000, "file_size": 33507070, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4363638646, "largest_seqno": 4426239552, "table_properties": {"data_size": 27595152, "index_size": 1790784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26178208, "raw_average_key_size": 28, "raw_value_size": 108039592, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934936, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391574, "oldest_key_time": 0, "file_creation_time": 1732391574, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291000, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:58.119771 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst) /dev/shm/ToplingTemp/Topling-262866-Hi7PkV 2024/11/24-03:52:58.969453 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst): first pass time = 0.85's, 157.970'MB/sec 2024/11/24-03:53:00.382059 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst): old prealloc_size = 147639500, real_size = 31471456 2024/11/24-03:53:00.382740 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919505 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.97 avg-zval = 32.06 2024/11/24-03:53:00.383578 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:01.221347 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291001.sst): second pass time = 1.41's, 76.859'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.359'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2690.285'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.15's, 243.291'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26741.688'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.859'MB/sec zip pipeline throughput = 76.859'MB/sec entries = 919505 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.97 avg-zval = 32.06 usrkeys = 919505 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.97 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919505 bytes = 3218288 } seq expand size = 6436535 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4440 value = 3.6794 dict = 1.00 all = 3.7754 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2252.816917 GB avg = 0.200 KB (by entry num) total key len = 307.220946 GB avg = 0.027 KB total ukey len = 237.596144 GB avg = 0.015 KB total ukey num = 15.750732020 Billion total entry num = 11.292045968 Billion write speed all = 37.499269244 MB/sec (with seq num) write speed all = 36.176027965 MB/sec (without seq num) 2024/11/24-03:53:01.264386 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291001: 919505 keys, 35550660 bytes, temperature: kWarm 2024/11/24-03:53:01.264453 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391581264419, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291001, "file_size": 35550660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4426238708, "table_properties": {"data_size": 29480592, "index_size": 1760832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25746140, "raw_average_key_size": 28, "raw_value_size": 108471590, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919505, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391578, "oldest_key_time": 0, "file_creation_time": 1732391578, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291001, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:01.299848 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst) /dev/shm/ToplingTemp/Topling-262866-Fa71KU 2024/11/24-03:53:02.291075 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst): first pass time = 0.99's, 135.411'MB/sec 2024/11/24-03:53:03.844319 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst): old prealloc_size = 147639500, real_size = 30683696 2024/11/24-03:53:03.844873 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 116.04 avg-zval = 30.78 2024/11/24-03:53:03.845632 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:04.671847 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291002.sst): second pass time = 1.55's, 69.650'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.049'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.900'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 275.415'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30301.543'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.650'MB/sec zip pipeline throughput = 101.770'MB/sec entries = 931829 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.04 avg-zval = 30.78 usrkeys = 931829 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.04 avg-zval = 30.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931829 bytes = 3261424 } seq expand size = 6522803 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.5152 value = 3.7703 dict = 1.00 all = 3.8573 } Zip/UnZip{ index = 0.0951 value = 0.2652 dict = 1.00 all = 0.2593 } ---------------------------- total value len = 2252.931566 GB avg = 0.199 KB (by entry num) total key len = 307.247038 GB avg = 0.027 KB total ukey len = 237.614781 GB avg = 0.015 KB total ukey num = 15.751663849 Billion total entry num = 11.292977797 Billion write speed all = 37.499435480 MB/sec (with seq num) write speed all = 36.176151888 MB/sec (without seq num) 2024/11/24-03:53:04.715167 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291002: 931829 keys, 34796006 bytes, temperature: kWarm 2024/11/24-03:53:04.715243 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391584715202, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291002, "file_size": 34796006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4426239566, "table_properties": {"data_size": 28678176, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26091212, "raw_average_key_size": 28, "raw_value_size": 108126598, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391581, "oldest_key_time": 0, "file_creation_time": 1732391581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291002, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:04.751520 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst) /dev/shm/ToplingTemp/Topling-262866-aMjE5S 2024/11/24-03:53:05.757761 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst): first pass time = 1.01's, 133.391'MB/sec 2024/11/24-03:53:07.208332 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst): old prealloc_size = 147639500, real_size = 31390800 2024/11/24-03:53:07.208891 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.98 2024/11/24-03:53:07.209640 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:08.026425 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291003.sst): second pass time = 1.45's, 74.845'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.215'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3312.087'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 278.632'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 37625.620'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.845'MB/sec zip pipeline throughput = 74.845'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.98 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4455 value = 3.6896 dict = 1.00 all = 3.7965 } Zip/UnZip{ index = 0.0957 value = 0.2710 dict = 1.00 all = 0.2634 } ---------------------------- total value len = 2253.046478 GB avg = 0.199 KB (by entry num) total key len = 307.272778 GB avg = 0.027 KB total ukey len = 237.633167 GB avg = 0.015 KB total ukey num = 15.752583149 Billion total entry num = 11.293897097 Billion write speed all = 37.499653103 MB/sec (with seq num) write speed all = 36.176326812 MB/sec (without seq num) 2024/11/24-03:53:08.070502 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291003: 919300 keys, 35352626 bytes, temperature: kWarm 2024/11/24-03:53:08.070575 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391588070537, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291003, "file_size": 35352626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4271013793, "largest_seqno": 4379240626, "table_properties": {"data_size": 29400576, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391584, "oldest_key_time": 0, "file_creation_time": 1732391584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291003, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:08.107557 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst) /dev/shm/ToplingTemp/Topling-262866-e2DgiV 2024/11/24-03:53:08.992352 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst): first pass time = 0.88's, 151.700'MB/sec 2024/11/24-03:53:10.428580 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst): old prealloc_size = 147639500, real_size = 31621056 2024/11/24-03:53:10.429117 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930898 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0296 GB avg-val = 116.18 avg-zval = 31.80 2024/11/24-03:53:10.429875 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:11.266825 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291004.sst): second pass time = 1.44's, 75.340'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.627'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3464.334'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 283.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35370.745'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.340'MB/sec zip pipeline throughput = 75.340'MB/sec entries = 930898 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.18 avg-zval = 31.80 usrkeys = 930898 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.18 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930898 bytes = 3258160 } seq expand size = 6516286 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4439 value = 3.6531 dict = 1.00 all = 3.7564 } Zip/UnZip{ index = 0.0957 value = 0.2737 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 2253.161147 GB avg = 0.199 KB (by entry num) total key len = 307.298843 GB avg = 0.027 KB total ukey len = 237.651785 GB avg = 0.015 KB total ukey num = 15.753514047 Billion total entry num = 11.294827995 Billion write speed all = 37.499934597 MB/sec (with seq num) write speed all = 36.176562039 MB/sec (without seq num) 2024/11/24-03:53:11.311731 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291004: 930898 keys, 35730758 bytes, temperature: kWarm 2024/11/24-03:53:11.311803 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391591311769, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291004, "file_size": 35730758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4426239580, "table_properties": {"data_size": 29605488, "index_size": 1782656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065144, "raw_average_key_size": 28, "raw_value_size": 108152624, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930898, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391588, "oldest_key_time": 0, "file_creation_time": 1732391588, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291004, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:11.350060 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst) /dev/shm/ToplingTemp/Topling-262866-bOh5uU 2024/11/24-03:53:12.349141 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst): first pass time = 1.00's, 134.347'MB/sec 2024/11/24-03:53:13.801367 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst): old prealloc_size = 147639500, real_size = 31372240 2024/11/24-03:53:13.801904 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-03:53:13.802703 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:14.622601 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291005.sst): second pass time = 1.45's, 74.763'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.026'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3452.826'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 279.111'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31769.098'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.763'MB/sec zip pipeline throughput = 74.763'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6920 dict = 1.00 all = 3.7985 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2253.276060 GB avg = 0.199 KB (by entry num) total key len = 307.324584 GB avg = 0.027 KB total ukey len = 237.670171 GB avg = 0.015 KB total ukey num = 15.754433347 Billion total entry num = 11.295747295 Billion write speed all = 37.500151504 MB/sec (with seq num) write speed all = 36.176736278 MB/sec (without seq num) 2024/11/24-03:53:14.666019 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291005: 919300 keys, 35334066 bytes, temperature: kWarm 2024/11/24-03:53:14.666089 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391594666054, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291005, "file_size": 35334066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279486295, "largest_seqno": 4388317404, "table_properties": {"data_size": 29382016, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391591, "oldest_key_time": 0, "file_creation_time": 1732391591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291005, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:14.703711 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst) /dev/shm/ToplingTemp/Topling-262866-pvreTS 2024/11/24-03:53:15.704291 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst): first pass time = 1.00's, 134.145'MB/sec 2024/11/24-03:53:17.020277 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst): old prealloc_size = 147639500, real_size = 32047504 2024/11/24-03:53:17.020833 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930938 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.17 avg-zval = 32.26 2024/11/24-03:53:17.022571 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:17.865186 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291006.sst): second pass time = 1.32's, 82.231'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.796'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3370.892'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 279.250'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 23680.198'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.231'MB/sec zip pipeline throughput = 82.231'MB/sec entries = 930938 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.17 avg-zval = 32.26 usrkeys = 930938 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.17 avg-zval = 32.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930938 bytes = 3258304 } seq expand size = 6516566 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4448 value = 3.6012 dict = 1.00 all = 3.7120 } Zip/UnZip{ index = 0.0957 value = 0.2777 dict = 1.00 all = 0.2694 } ---------------------------- total value len = 2253.390728 GB avg = 0.199 KB (by entry num) total key len = 307.350650 GB avg = 0.027 KB total ukey len = 237.688790 GB avg = 0.015 KB total ukey num = 15.755364285 Billion total entry num = 11.296678233 Billion write speed all = 37.500431760 MB/sec (with seq num) write speed all = 36.176970312 MB/sec (without seq num) 2024/11/24-03:53:17.909785 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291006: 930938 keys, 36157490 bytes, temperature: kWarm 2024/11/24-03:53:17.909857 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391597909820, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291006, "file_size": 36157490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4426239600, "table_properties": {"data_size": 30032000, "index_size": 1782592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066264, "raw_average_key_size": 28, "raw_value_size": 108151464, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930938, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391594, "oldest_key_time": 0, "file_creation_time": 1732391594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291006, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:17.951535 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst) /dev/shm/ToplingTemp/Topling-262866-GM0ypV 2024/11/24-03:53:18.957060 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst): first pass time = 1.01's, 133.485'MB/sec 2024/11/24-03:53:20.251624 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst): old prealloc_size = 147639500, real_size = 31399632 2024/11/24-03:53:20.252188 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-03:53:20.252947 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:21.070100 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291007.sst): second pass time = 1.29's, 83.843'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.205'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.608'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 275.052'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35251.633'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.843'MB/sec zip pipeline throughput = 83.843'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4451 value = 3.6885 dict = 1.00 all = 3.7956 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2253.505640 GB avg = 0.199 KB (by entry num) total key len = 307.376390 GB avg = 0.027 KB total ukey len = 237.707176 GB avg = 0.015 KB total ukey num = 15.756283585 Billion total entry num = 11.297597533 Billion write speed all = 37.500731474 MB/sec (with seq num) write speed all = 36.177224444 MB/sec (without seq num) 2024/11/24-03:53:21.113084 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291007: 919300 keys, 35361458 bytes, temperature: kWarm 2024/11/24-03:53:21.113157 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391601113120, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291007, "file_size": 35361458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4287955877, "largest_seqno": 4397555154, "table_properties": {"data_size": 29409344, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391597, "oldest_key_time": 0, "file_creation_time": 1732391597, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291007, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:21.151230 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst) /dev/shm/ToplingTemp/Topling-262866-XmmwpU 2024/11/24-03:53:21.299759 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst): first pass time = 0.15's, 235.312'MB/sec 2024/11/24-03:53:21.641807 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst): old prealloc_size = 147639500, real_size = 9721072 2024/11/24-03:53:21.642029 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 239476 prefix = 4 raw-key = 0.0038 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0282 GB zip-val = 0.0092 GB avg-val = 117.91 avg-zval = 38.41 2024/11/24-03:53:21.642292 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.035 GB 2024/11/24-03:53:21.857547 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00/291008.sst): second pass time = 0.34's, 82.714'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 49.745'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2281.229'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.329'MB/sec write SST data time = 0.03's, 318.923'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.290'MB, throughput = 9586.143'MB/sec zip my value time = 0.34's, unzip length = 0.028'GB zip my value throughput = 82.714'MB/sec zip pipeline throughput = 82.714'MB/sec entries = 239476 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.91 avg-zval = 38.41 usrkeys = 239476 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.91 avg-zval = 38.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 239476 bytes = 778304 } seq expand size = 1676332 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0048 GB value = 0.0282 GB dict = 0.29 MB all = 0.0349 GB } __ZipSize{ index = 0.0005 GB value = 0.0092 GB dict = 0.29 MB all = 0.0107 GB } UnZip/Zip{ index = 10.3508 value = 3.0697 dict = 1.00 all = 3.2561 } Zip/UnZip{ index = 0.0966 value = 0.3258 dict = 1.00 all = 0.3071 } ---------------------------- total value len = 2253.535553 GB avg = 0.199 KB (by entry num) total key len = 307.383096 GB avg = 0.027 KB total ukey len = 237.711965 GB avg = 0.015 KB total ukey num = 15.756523061 Billion total entry num = 11.297837009 Billion write speed all = 37.500835260 MB/sec (with seq num) write speed all = 36.177315437 MB/sec (without seq num) 2024/11/24-03:53:21.876569 139957676762688 [compaction_job.cc:1948] [default] [JOB 41097] Generated table #291008: 239476 keys, 10730953 bytes, temperature: kWarm 2024/11/24-03:53:21.876625 139957676762688 EVENT_LOG_v1 {"time_micros": 1732391601876595, "cf_name": "default", "job": 41097, "event": "table_file_creation", "file_number": 291008, "file_size": 10730953, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4379158070, "largest_seqno": 4426239612, "table_properties": {"data_size": 9198304, "index_size": 462720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6705328, "raw_average_key_size": 28, "raw_value_size": 28235852, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 239476, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391601, "oldest_key_time": 0, "file_creation_time": 1732391601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291008, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:21.890021 139957676762688 [compaction_job.cc:703] job-41097: subcompact[0], size: 0.683694 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41097/att-00 [290989,290990,290991,290992,290993,290994,290995,290996,290997,290998,290999,291000,291001,291002,291003,291004,291005,291006,291007,291008] 2024/11/24-03:53:22.007705 139957676762688 (Original Log Time 2024/11/24-03:52:17.098026) EVENT_LOG_v1 {"time_micros": 1732391537097983, "job": 41097, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290947], "files_L3": [290977, 290978, 290922, 290923, 290924, 290890, 290891, 290892, 290893, 290894, 290895, 290896, 290897, 290898, 290899, 290900, 290901, 290902, 290903, 290904, 290979], "score": 0.63582, "input_data_size": 713628881, "oldest_snapshot_seqno": 4426433088} 2024/11/24-03:53:22.007709 139957676762688 (Original Log Time 2024/11/24-03:53:22.006090) [compaction_job.cc:2022] [default] [JOB 41097] Compacted 1@2 + 21@3 files to L3 => 683693731 bytes 2024/11/24-03:53:22.007711 139957676762688 (Original Log Time 2024/11/24-03:53:22.007633) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.6 wr, level 3, files in(1, 21) out(20 +0 blob) MB in(27.1, 653.5 +0.0 blob) out(652.0 +0.0 blob), read-write-amplify(49.2) write-amplify(24.1) OK, records in: 17959730, records dropped: 97832 output_compression: Snappy 2024/11/24-03:53:22.007714 139957676762688 (Original Log Time 2024/11/24-03:53:22.007679) EVENT_LOG_v1 {"time_micros": 1732391602007658, "cf": "default", "job": 41097, "event": "compaction_finished", "compaction_time_micros": 64792001, "compaction_time_cpu_micros": 26944005, "output_level": 3, "num_output_files": 20, "total_output_size": 683693731, "num_input_records": 17959730, "num_output_records": 17861898, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 475755513, "file_range_sync_nanos": 0, "file_fsync_nanos": 14377, "file_prepare_write_nanos": 45856, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-03:53:22.142371 139957676762688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732391536,"executesMs":65042,"compactionJobId":41097,"attempt":0,"compactionInputRawBytes":2598971384,"compactionInputZipBytes":713628881,"compactionOutputRawBytes":0,"compactionOutputZipBytes":683693731,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-66b504c3ae50047eb16b5aa5c2c13489-60f0489f926e84e3-00"}}, response = 2024/11/24-03:53:22.142466 139957676762688 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41097/att-00: olev 3, work 65.042 s, result 7.704 ms, install 1.639 ms, input{raw 2.420 GiB zip 680.570 MiB}