2024/11/24-05:35:59.623484 139957215163968 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00: fork to child time = 0.007742 sec 2024/11/24-05:35:59.624298 139957215163968 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00 2024/11/24-05:35:59.630037 139957215163968 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00 2024/11/24-05:35:59.630931 139957215163968 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:35:59.631178 139957215163968 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:35:59.682310 139957215163968 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42285/att-00/MANIFEST-206347 2024/11/24-05:35:59.697612 139957215163968 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42285/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 293726, last_sequence is 4462363781, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 293584 2024/11/24-05:35:59.697619 139957215163968 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:35:59.697757 139957215163968 [version_set.cc:5721] Creating manifest 293726 2024/11/24-05:35:59.731492 139957215163968 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:35:59.751397 139957215163968 [compaction_job.cc:2437] [default] [JOB 42285] Compacting 1@2 + 24@3 files to L3, score 0.90, subcompactions 1 : 1 2024/11/24-05:35:59.751417 139957215163968 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31704 Base level 2, inputs: [293673(27MB)], [293724(18MB) 293629(34MB) 293630(35MB) 293631(32MB) 293632(30MB) 293633(35MB) 293634(33MB) 293635(31MB) 293636(33MB) 293637(32MB) 293638(35MB) 293639(33MB) 293640(16MB) 293600(34MB) 293601(33MB) 293602(35MB) 293603(33MB) 293604(35MB) 293605(33MB) 293606(33MB) 293607(33MB) 293608(32MB) 293609(34MB) 293690(33MB)] 2024/11/24-05:35:59.759568 139957215163968 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:35:59.759597 139957215163968 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:35:59.759617 139957215163968 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:35:59.759622 139957215163968 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:35:59.759635 139957215163968 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:36:00.186747 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst) /dev/shm/ToplingTemp/Topling-281184-3sppG0 2024/11/24-05:36:01.087877 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst): first pass time = 0.90's, 148.951'MB/sec 2024/11/24-05:36:02.632234 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst): old prealloc_size = 147639500, real_size = 33234352 2024/11/24-05:36:02.632941 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927226 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.75 avg-zval = 33.68 2024/11/24-05:36:02.633841 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:03.521507 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293727.sst): second pass time = 1.54's, 70.167'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.983'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2605.914'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.15's, 241.156'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35297.446'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.167'MB/sec zip pipeline throughput = 70.167'MB/sec entries = 927226 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 33.68 usrkeys = 927226 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927226 bytes = 3245312 } seq expand size = 6490582 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4463 value = 3.4667 dict = 1.00 all = 3.5950 } Zip/UnZip{ index = 0.0957 value = 0.2885 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 2251.440319 GB avg = 0.200 KB (by entry num) total key len = 306.908925 GB avg = 0.027 KB total ukey len = 237.373272 GB avg = 0.015 KB total ukey num = 15.739588389 Billion total entry num = 11.280902337 Billion write speed all = 34.362717758 MB/sec (with seq num) write speed all = 33.150553425 MB/sec (without seq num) 2024/11/24-05:36:03.568965 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293727: 927226 keys, 37334372 bytes, temperature: kWarm 2024/11/24-05:36:03.569079 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397763569019, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293727, "file_size": 37334372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4461975910, "table_properties": {"data_size": 31227104, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962328, "raw_average_key_size": 28, "raw_value_size": 108255416, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927226, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397760, "oldest_key_time": 0, "file_creation_time": 1732397760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293727, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:03.613957 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst) /dev/shm/ToplingTemp/Topling-281184-9TmGn0 2024/11/24-05:36:04.644738 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst): first pass time = 1.03's, 130.225'MB/sec 2024/11/24-05:36:06.086923 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst): old prealloc_size = 147639500, real_size = 31306016 2024/11/24-05:36:06.087463 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920964 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.74 avg-zval = 31.83 2024/11/24-05:36:06.088219 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:06.914330 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293728.sst): second pass time = 1.44's, 75.230'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3427.976'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 266.434'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34752.076'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.230'MB/sec zip pipeline throughput = 75.230'MB/sec entries = 920964 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.74 avg-zval = 31.83 usrkeys = 920964 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.74 avg-zval = 31.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920964 bytes = 3223392 } seq expand size = 6446748 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4442 value = 3.6992 dict = 1.00 all = 3.7928 } Zip/UnZip{ index = 0.0957 value = 0.2703 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2251.555197 GB avg = 0.200 KB (by entry num) total key len = 306.934712 GB avg = 0.027 KB total ukey len = 237.391691 GB avg = 0.015 KB total ukey num = 15.740509353 Billion total entry num = 11.281823301 Billion write speed all = 34.363041155 MB/sec (with seq num) write speed all = 33.150833103 MB/sec (without seq num) 2024/11/24-05:36:06.960988 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293728: 920964 keys, 35387072 bytes, temperature: kWarm 2024/11/24-05:36:06.961058 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397766961023, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293728, "file_size": 35387072, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4320580777, "largest_seqno": 4461973555, "table_properties": {"data_size": 29312016, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25786992, "raw_average_key_size": 28, "raw_value_size": 108430760, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920964, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397763, "oldest_key_time": 0, "file_creation_time": 1732397763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293728, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:06.996206 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst) /dev/shm/ToplingTemp/Topling-281184-MZnEKZ 2024/11/24-05:36:07.941631 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst): first pass time = 0.95's, 141.971'MB/sec 2024/11/24-05:36:09.487699 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst): old prealloc_size = 147639500, real_size = 33165776 2024/11/24-05:36:09.488249 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.95 avg-zval = 33.65 2024/11/24-05:36:09.489025 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:10.337792 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293729.sst): second pass time = 1.55's, 70.077'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.295'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 281.467'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28935.530'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.077'MB/sec zip pipeline throughput = 93.627'MB/sec entries = 925990 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 33.65 usrkeys = 925990 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925990 bytes = 3240976 } seq expand size = 6481930 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4463 value = 3.4752 dict = 1.00 all = 3.6017 } Zip/UnZip{ index = 0.0957 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2251.669969 GB avg = 0.200 KB (by entry num) total key len = 306.960640 GB avg = 0.027 KB total ukey len = 237.410211 GB avg = 0.015 KB total ukey num = 15.741435343 Billion total entry num = 11.282749291 Billion write speed all = 34.363350835 MB/sec (with seq num) write speed all = 33.151099028 MB/sec (without seq num) 2024/11/24-05:36:10.385287 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293729: 925990 keys, 37265156 bytes, temperature: kWarm 2024/11/24-05:36:10.385360 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397770385327, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293729, "file_size": 37265156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4461974355, "table_properties": {"data_size": 31161216, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927720, "raw_average_key_size": 28, "raw_value_size": 108290112, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397766, "oldest_key_time": 0, "file_creation_time": 1732397766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293729, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:10.421871 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst) /dev/shm/ToplingTemp/Topling-281184-gVADh3 2024/11/24-05:36:11.271058 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst): first pass time = 0.85's, 158.061'MB/sec 2024/11/24-05:36:12.831536 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst): old prealloc_size = 147639500, real_size = 28011168 2024/11/24-05:36:12.832075 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944019 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1078 GB zip-val = 0.0260 GB avg-val = 114.18 avg-zval = 27.51 2024/11/24-05:36:12.832873 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:13.677228 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293730.sst): second pass time = 1.56's, 69.150'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 38.026'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3477.269'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 239.788'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 22880.465'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.150'MB/sec zip pipeline throughput = 135.099'MB/sec entries = 944019 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.18 avg-zval = 27.51 usrkeys = 944019 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.18 avg-zval = 27.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944019 bytes = 3304080 } seq expand size = 6608133 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0260 GB dict = 1.07 MB all = 0.0322 GB } UnZip/Zip{ index = 10.4438 value = 4.1508 dict = 1.00 all = 4.1738 } Zip/UnZip{ index = 0.0958 value = 0.2409 dict = 1.00 all = 0.2396 } ---------------------------- total value len = 2251.784362 GB avg = 0.200 KB (by entry num) total key len = 306.987072 GB avg = 0.027 KB total ukey len = 237.429091 GB avg = 0.015 KB total ukey num = 15.742379362 Billion total entry num = 11.283693310 Billion write speed all = 34.363700982 MB/sec (with seq num) write speed all = 33.151402118 MB/sec (without seq num) 2024/11/24-05:36:13.734075 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293730: 944019 keys, 32157106 bytes, temperature: kWarm 2024/11/24-05:36:13.734122 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397773734094, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293730, "file_size": 32157106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4461973849, "table_properties": {"data_size": 25967184, "index_size": 1807808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26432532, "raw_average_key_size": 28, "raw_value_size": 107785306, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944019, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397770, "oldest_key_time": 0, "file_creation_time": 1732397770, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293730, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:13.775096 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst) /dev/shm/ToplingTemp/Topling-281184-aGuOWZ 2024/11/24-05:36:14.798514 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst): first pass time = 1.02's, 131.151'MB/sec 2024/11/24-05:36:16.072049 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst): old prealloc_size = 147639500, real_size = 28074624 2024/11/24-05:36:16.072621 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 944559 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0260 GB avg-val = 114.10 avg-zval = 27.57 2024/11/24-05:36:16.073398 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:16.904869 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293731.sst): second pass time = 1.27's, 84.699'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.694'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.238'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.13's, 251.504'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.074'MB, throughput = 30022.706'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.699'MB/sec zip pipeline throughput = 84.699'MB/sec entries = 944559 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.10 avg-zval = 27.57 usrkeys = 944559 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.10 avg-zval = 27.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 944559 bytes = 3305968 } seq expand size = 6611913 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0260 GB dict = 1.07 MB all = 0.0322 GB } UnZip/Zip{ index = 10.5130 value = 4.1384 dict = 1.00 all = 4.1652 } Zip/UnZip{ index = 0.0951 value = 0.2416 dict = 1.00 all = 0.2401 } ---------------------------- total value len = 2251.898744 GB avg = 0.200 KB (by entry num) total key len = 307.013520 GB avg = 0.027 KB total ukey len = 237.447982 GB avg = 0.015 KB total ukey num = 15.743323921 Billion total entry num = 11.284637869 Billion write speed all = 34.364102727 MB/sec (with seq num) write speed all = 33.151754932 MB/sec (without seq num) 2024/11/24-05:36:16.945443 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293731: 944559 keys, 32223994 bytes, temperature: kWarm 2024/11/24-05:36:16.945516 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397776945478, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293731, "file_size": 32223994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4461972283, "table_properties": {"data_size": 26041392, "index_size": 1796928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26447652, "raw_average_key_size": 28, "raw_value_size": 107770194, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 944559, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397773, "oldest_key_time": 0, "file_creation_time": 1732397773, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293731, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:16.981630 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst) /dev/shm/ToplingTemp/Topling-281184-dArSO0 2024/11/24-05:36:17.973539 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst): first pass time = 0.99's, 135.318'MB/sec 2024/11/24-05:36:19.302228 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst): old prealloc_size = 147639500, real_size = 31509024 2024/11/24-05:36:19.302795 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.11 2024/11/24-05:36:19.303561 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:20.123797 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293732.sst): second pass time = 1.33's, 81.714'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.315'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3247.933'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 271.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31460.508'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.714'MB/sec zip pipeline throughput = 81.714'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.11 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4439 value = 3.6749 dict = 1.00 all = 3.7717 } Zip/UnZip{ index = 0.0957 value = 0.2721 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 2252.013657 GB avg = 0.200 KB (by entry num) total key len = 307.039260 GB avg = 0.027 KB total ukey len = 237.466368 GB avg = 0.015 KB total ukey num = 15.744243221 Billion total entry num = 11.285557169 Billion write speed all = 34.364506084 MB/sec (with seq num) write speed all = 33.152111935 MB/sec (without seq num) 2024/11/24-05:36:20.168220 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293732: 919300 keys, 35585714 bytes, temperature: kWarm 2024/11/24-05:36:20.168290 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397780168254, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293732, "file_size": 35585714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268759135, "largest_seqno": 4412489040, "table_properties": {"data_size": 29518544, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397776, "oldest_key_time": 0, "file_creation_time": 1732397776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293732, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:20.205505 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst) /dev/shm/ToplingTemp/Topling-281184-8ONlc1 2024/11/24-05:36:21.080424 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst): first pass time = 0.87's, 153.413'MB/sec 2024/11/24-05:36:22.379952 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst): old prealloc_size = 147639500, real_size = 29326640 2024/11/24-05:36:22.380526 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936789 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0273 GB avg-val = 115.27 avg-zval = 29.14 2024/11/24-05:36:22.382504 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:23.207928 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293733.sst): second pass time = 1.30's, 83.180'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3442.461'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 256.781'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34210.744'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.180'MB/sec zip pipeline throughput = 83.180'MB/sec entries = 936789 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.27 avg-zval = 29.14 usrkeys = 936789 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.27 avg-zval = 29.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936789 bytes = 3161680 } seq expand size = 6557523 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0273 GB dict = 1.08 MB all = 0.0333 GB } UnZip/Zip{ index = 10.4426 value = 3.9559 dict = 1.00 all = 4.0263 } Zip/UnZip{ index = 0.0958 value = 0.2528 dict = 1.00 all = 0.2484 } ---------------------------- total value len = 2252.128202 GB avg = 0.200 KB (by entry num) total key len = 307.065490 GB avg = 0.027 KB total ukey len = 237.485104 GB avg = 0.015 KB total ukey num = 15.745180010 Billion total entry num = 11.286493958 Billion write speed all = 34.364973256 MB/sec (with seq num) write speed all = 33.152528683 MB/sec (without seq num) 2024/11/24-05:36:23.249396 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293733: 936789 keys, 33335048 bytes, temperature: kWarm 2024/11/24-05:36:23.249467 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397783249431, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293733, "file_size": 33335048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4338595157, "largest_seqno": 4461974722, "table_properties": {"data_size": 27298080, "index_size": 1794176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26230092, "raw_average_key_size": 28, "raw_value_size": 107987698, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936789, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397780, "oldest_key_time": 0, "file_creation_time": 1732397780, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293733, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:23.286442 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst) /dev/shm/ToplingTemp/Topling-281184-ZFj0hZ 2024/11/24-05:36:24.258420 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst): first pass time = 0.97's, 138.093'MB/sec 2024/11/24-05:36:25.586572 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst): old prealloc_size = 147639500, real_size = 32631872 2024/11/24-05:36:25.587151 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932806 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.89 avg-zval = 32.82 2024/11/24-05:36:25.587927 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:26.453085 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293734.sst): second pass time = 1.33's, 81.469'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3216.609'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 278.019'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32649.885'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.469'MB/sec zip pipeline throughput = 81.469'MB/sec entries = 932806 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.89 avg-zval = 32.82 usrkeys = 932806 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.89 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932806 bytes = 3264832 } seq expand size = 6529642 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4455 value = 3.5312 dict = 1.00 all = 3.6525 } Zip/UnZip{ index = 0.0957 value = 0.2832 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 2252.242831 GB avg = 0.200 KB (by entry num) total key len = 307.091609 GB avg = 0.027 KB total ukey len = 237.503760 GB avg = 0.015 KB total ukey num = 15.746112816 Billion total entry num = 11.287426764 Billion write speed all = 34.365365711 MB/sec (with seq num) write speed all = 33.152873767 MB/sec (without seq num) 2024/11/24-05:36:26.497595 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293734: 932806 keys, 36747230 bytes, temperature: kWarm 2024/11/24-05:36:26.497668 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397786497629, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293734, "file_size": 36747230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4461974200, "table_properties": {"data_size": 30612464, "index_size": 1786048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26118568, "raw_average_key_size": 28, "raw_value_size": 108099252, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932806, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397783, "oldest_key_time": 0, "file_creation_time": 1732397783, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293734, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:26.543405 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst) /dev/shm/ToplingTemp/Topling-281184-zcA6LY 2024/11/24-05:36:27.420528 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst): first pass time = 0.88's, 153.028'MB/sec 2024/11/24-05:36:28.723062 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst): old prealloc_size = 147639500, real_size = 31324944 2024/11/24-05:36:28.723596 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-05:36:28.724500 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:29.537372 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293735.sst): second pass time = 1.30's, 83.338'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.335'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3427.868'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.030'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31414.054'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.338'MB/sec zip pipeline throughput = 83.338'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4458 value = 3.6979 dict = 1.00 all = 3.8036 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2252.357743 GB avg = 0.200 KB (by entry num) total key len = 307.117349 GB avg = 0.027 KB total ukey len = 237.522146 GB avg = 0.015 KB total ukey num = 15.747032116 Billion total entry num = 11.288346064 Billion write speed all = 34.365831091 MB/sec (with seq num) write speed all = 33.153290613 MB/sec (without seq num) 2024/11/24-05:36:29.580615 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293735: 919300 keys, 35286770 bytes, temperature: kWarm 2024/11/24-05:36:29.580686 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397789580649, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293735, "file_size": 35286770, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4320002050, "largest_seqno": 4438940125, "table_properties": {"data_size": 29334784, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397786, "oldest_key_time": 0, "file_creation_time": 1732397786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293735, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:29.621211 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst) /dev/shm/ToplingTemp/Topling-281184-TLjA10 2024/11/24-05:36:30.570943 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst): first pass time = 0.95's, 141.328'MB/sec 2024/11/24-05:36:31.867816 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst): old prealloc_size = 147639500, real_size = 30649312 2024/11/24-05:36:31.868388 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936735 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0286 GB avg-val = 115.28 avg-zval = 30.57 2024/11/24-05:36:31.869161 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:32.711709 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293736.sst): second pass time = 1.30's, 83.322'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.211'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.107'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 265.183'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34569.860'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.322'MB/sec zip pipeline throughput = 83.322'MB/sec entries = 936735 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.28 avg-zval = 30.57 usrkeys = 936735 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.28 avg-zval = 30.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936735 bytes = 3278592 } seq expand size = 6557145 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.5106 value = 3.7716 dict = 1.00 all = 3.8594 } Zip/UnZip{ index = 0.0951 value = 0.2651 dict = 1.00 all = 0.2591 } ---------------------------- total value len = 2252.472290 GB avg = 0.200 KB (by entry num) total key len = 307.143578 GB avg = 0.027 KB total ukey len = 237.540881 GB avg = 0.015 KB total ukey num = 15.747968851 Billion total entry num = 11.289282799 Billion write speed all = 34.366256521 MB/sec (with seq num) write speed all = 33.153667106 MB/sec (without seq num) 2024/11/24-05:36:32.756059 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293736: 936735 keys, 34776624 bytes, temperature: kWarm 2024/11/24-05:36:32.756131 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397792756096, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293736, "file_size": 34776624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4461976287, "table_properties": {"data_size": 28632464, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26228580, "raw_average_key_size": 28, "raw_value_size": 107989194, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936735, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397789, "oldest_key_time": 0, "file_creation_time": 1732397789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293736, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:32.791057 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst) /dev/shm/ToplingTemp/Topling-281184-7lmAc2 2024/11/24-05:36:33.786920 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst): first pass time = 1.00's, 134.779'MB/sec 2024/11/24-05:36:35.083143 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst): old prealloc_size = 147639500, real_size = 31401408 2024/11/24-05:36:35.083685 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.98 2024/11/24-05:36:35.084424 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:35.897501 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293737.sst): second pass time = 1.30's, 83.764'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3394.739'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.038'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33887.501'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.764'MB/sec zip pipeline throughput = 83.764'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.98 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3768 value = 3.6898 dict = 1.00 all = 3.7954 } Zip/UnZip{ index = 0.0964 value = 0.2710 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2252.587202 GB avg = 0.200 KB (by entry num) total key len = 307.169318 GB avg = 0.027 KB total ukey len = 237.559267 GB avg = 0.015 KB total ukey num = 15.748888151 Billion total entry num = 11.290202099 Billion write speed all = 34.366674990 MB/sec (with seq num) write speed all = 33.154038701 MB/sec (without seq num) 2024/11/24-05:36:35.940968 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293737: 919300 keys, 35363250 bytes, temperature: kWarm 2024/11/24-05:36:35.941061 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397795941024, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293737, "file_size": 35363250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280087283, "largest_seqno": 4390180614, "table_properties": {"data_size": 29399536, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397792, "oldest_key_time": 0, "file_creation_time": 1732397792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293737, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:35.975272 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst) /dev/shm/ToplingTemp/Topling-281184-kPTOq1 2024/11/24-05:36:36.732274 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst): first pass time = 0.76's, 177.313'MB/sec 2024/11/24-05:36:38.043878 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst): old prealloc_size = 147639500, real_size = 30384400 2024/11/24-05:36:38.044463 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928801 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.51 avg-zval = 30.55 2024/11/24-05:36:38.045200 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:38.864238 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293738.sst): second pass time = 1.31's, 82.552'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.898'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.332'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 262.095'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25809.811'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.552'MB/sec zip pipeline throughput = 82.552'MB/sec entries = 928801 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 30.55 usrkeys = 928801 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.51 avg-zval = 30.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928801 bytes = 3134720 } seq expand size = 6501607 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4455 value = 3.8138 dict = 1.00 all = 3.9050 } Zip/UnZip{ index = 0.0957 value = 0.2622 dict = 1.00 all = 0.2561 } ---------------------------- total value len = 2252.701915 GB avg = 0.200 KB (by entry num) total key len = 307.195325 GB avg = 0.027 KB total ukey len = 237.577843 GB avg = 0.015 KB total ukey num = 15.749816952 Billion total entry num = 11.291130900 Billion write speed all = 34.367195386 MB/sec (with seq num) write speed all = 33.154507640 MB/sec (without seq num) 2024/11/24-05:36:38.906590 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293738: 928801 keys, 34370572 bytes, temperature: kWarm 2024/11/24-05:36:38.906661 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397798906626, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293738, "file_size": 34370572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4390180615, "largest_seqno": 4461976041, "table_properties": {"data_size": 28373632, "index_size": 1778368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006428, "raw_average_key_size": 28, "raw_value_size": 108211414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928801, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397795, "oldest_key_time": 0, "file_creation_time": 1732397795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293738, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:38.950946 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst) /dev/shm/ToplingTemp/Topling-281184-60zF90 2024/11/24-05:36:39.876868 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst): first pass time = 0.93's, 144.965'MB/sec 2024/11/24-05:36:41.193324 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst): old prealloc_size = 147639500, real_size = 32547904 2024/11/24-05:36:41.193868 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923268 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.37 avg-zval = 33.09 2024/11/24-05:36:41.194641 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:42.025746 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293739.sst): second pass time = 1.32's, 82.370'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.563'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 284.393'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32439.650'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.370'MB/sec zip pipeline throughput = 82.370'MB/sec entries = 923268 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.37 avg-zval = 33.09 usrkeys = 923268 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.37 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923268 bytes = 3231456 } seq expand size = 6462876 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4438 value = 3.5473 dict = 1.00 all = 3.6634 } Zip/UnZip{ index = 0.0958 value = 0.2819 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 2252.816744 GB avg = 0.200 KB (by entry num) total key len = 307.221176 GB avg = 0.027 KB total ukey len = 237.596308 GB avg = 0.015 KB total ukey num = 15.750740220 Billion total entry num = 11.292054168 Billion write speed all = 34.367625355 MB/sec (with seq num) write speed all = 33.154889922 MB/sec (without seq num) 2024/11/24-05:36:42.071145 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293739: 923268 keys, 36637152 bytes, temperature: kWarm 2024/11/24-05:36:42.071211 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397802071176, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293739, "file_size": 36637152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4461973113, "table_properties": {"data_size": 30548848, "index_size": 1768064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25851504, "raw_average_key_size": 28, "raw_value_size": 108366288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923268, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397798, "oldest_key_time": 0, "file_creation_time": 1732397798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293739, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:42.111332 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst) /dev/shm/ToplingTemp/Topling-281184-85XY31 2024/11/24-05:36:43.104917 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst): first pass time = 0.99's, 135.090'MB/sec 2024/11/24-05:36:44.416435 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst): old prealloc_size = 147639500, real_size = 31398832 2024/11/24-05:36:44.417003 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920852 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.75 avg-zval = 31.93 2024/11/24-05:36:44.418309 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:45.238910 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293740.sst): second pass time = 1.31's, 82.787'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.999'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.699'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 267.555'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31458.750'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.787'MB/sec zip pipeline throughput = 82.787'MB/sec entries = 920852 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.75 avg-zval = 31.93 usrkeys = 920852 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.75 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920852 bytes = 3107888 } seq expand size = 6445964 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4452 value = 3.6876 dict = 1.00 all = 3.7953 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2252.931624 GB avg = 0.199 KB (by entry num) total key len = 307.246960 GB avg = 0.027 KB total ukey len = 237.614726 GB avg = 0.015 KB total ukey num = 15.751661072 Billion total entry num = 11.292975020 Billion write speed all = 34.368031233 MB/sec (with seq num) write speed all = 33.155249217 MB/sec (without seq num) 2024/11/24-05:36:45.282607 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293740: 920852 keys, 35364564 bytes, temperature: kWarm 2024/11/24-05:36:45.282681 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397805282645, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293740, "file_size": 35364564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4356582680, "largest_seqno": 4461976053, "table_properties": {"data_size": 29405216, "index_size": 1763200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25783856, "raw_average_key_size": 28, "raw_value_size": 108433888, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920852, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397802, "oldest_key_time": 0, "file_creation_time": 1732397802, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:45.320524 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst) /dev/shm/ToplingTemp/Topling-281184-I3Ps9Z 2024/11/24-05:36:46.329080 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst): first pass time = 1.01's, 133.084'MB/sec 2024/11/24-05:36:47.677503 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst): old prealloc_size = 147639500, real_size = 32895376 2024/11/24-05:36:47.678088 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930225 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.29 avg-zval = 33.19 2024/11/24-05:36:47.680525 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:48.540187 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293741.sst): second pass time = 1.35's, 80.297'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.894'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.234'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 282.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31648.267'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.297'MB/sec zip pipeline throughput = 80.297'MB/sec entries = 930225 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.29 avg-zval = 33.19 usrkeys = 930225 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.29 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930225 bytes = 3255808 } seq expand size = 6511575 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3772 value = 3.5041 dict = 1.00 all = 3.6272 } Zip/UnZip{ index = 0.0964 value = 0.2854 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 2253.046307 GB avg = 0.199 KB (by entry num) total key len = 307.273006 GB avg = 0.027 KB total ukey len = 237.633330 GB avg = 0.015 KB total ukey num = 15.752591297 Billion total entry num = 11.293905245 Billion write speed all = 34.368397303 MB/sec (with seq num) write speed all = 33.155569136 MB/sec (without seq num) 2024/11/24-05:36:48.586022 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293741: 930225 keys, 37002846 bytes, temperature: kWarm 2024/11/24-05:36:48.586089 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397808586055, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293741, "file_size": 37002846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4461975204, "table_properties": {"data_size": 30869824, "index_size": 1792832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046300, "raw_average_key_size": 28, "raw_value_size": 108171430, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930225, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397805, "oldest_key_time": 0, "file_creation_time": 1732397805, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:48.623874 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst) /dev/shm/ToplingTemp/Topling-281184-EYosD1 2024/11/24-05:36:49.495870 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst): first pass time = 0.87's, 153.928'MB/sec 2024/11/24-05:36:50.807835 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst): old prealloc_size = 147639500, real_size = 31411360 2024/11/24-05:36:50.808460 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-05:36:50.809215 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:51.629955 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293742.sst): second pass time = 1.31's, 82.731'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.028'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2968.186'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 267.725'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34226.365'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.731'MB/sec zip pipeline throughput = 82.731'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4451 value = 3.6871 dict = 1.00 all = 3.7943 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2253.161220 GB avg = 0.199 KB (by entry num) total key len = 307.298747 GB avg = 0.027 KB total ukey len = 237.651716 GB avg = 0.015 KB total ukey num = 15.753510597 Billion total entry num = 11.294824545 Billion write speed all = 34.368859892 MB/sec (with seq num) write speed all = 33.155983308 MB/sec (without seq num) 2024/11/24-05:36:51.673029 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293742: 919300 keys, 35373170 bytes, temperature: kWarm 2024/11/24-05:36:51.673098 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397811673064, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293742, "file_size": 35373170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4327147687, "largest_seqno": 4438466221, "table_properties": {"data_size": 29421072, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397808, "oldest_key_time": 0, "file_creation_time": 1732397808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:51.715414 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst) /dev/shm/ToplingTemp/Topling-281184-Y8Uqs1 2024/11/24-05:36:52.621493 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst): first pass time = 0.91's, 148.137'MB/sec 2024/11/24-05:36:53.943515 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst): old prealloc_size = 147639500, real_size = 32363328 2024/11/24-05:36:53.944103 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931872 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.03 avg-zval = 32.58 2024/11/24-05:36:53.945532 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:54.799509 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293743.sst): second pass time = 1.32's, 81.833'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.613'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.179'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 271.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33233.330'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.833'MB/sec zip pipeline throughput = 81.833'MB/sec entries = 931872 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 32.58 usrkeys = 931872 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 32.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931872 bytes = 3261568 } seq expand size = 6523104 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5141 value = 3.5617 dict = 1.00 all = 3.6796 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 2253.275868 GB avg = 0.199 KB (by entry num) total key len = 307.324839 GB avg = 0.027 KB total ukey len = 237.670353 GB avg = 0.015 KB total ukey num = 15.754442469 Billion total entry num = 11.295756417 Billion write speed all = 34.369286815 MB/sec (with seq num) write speed all = 33.156361767 MB/sec (without seq num) 2024/11/24-05:36:54.844327 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293743: 931872 keys, 36475832 bytes, temperature: kWarm 2024/11/24-05:36:54.844400 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397814844364, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293743, "file_size": 36475832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4461976064, "table_properties": {"data_size": 30357552, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092416, "raw_average_key_size": 28, "raw_value_size": 108125436, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931872, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397811, "oldest_key_time": 0, "file_creation_time": 1732397811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:54.892429 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst) /dev/shm/ToplingTemp/Topling-281184-ZRaU11 2024/11/24-05:36:55.888913 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst): first pass time = 1.00's, 134.697'MB/sec 2024/11/24-05:36:57.194569 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst): old prealloc_size = 147639500, real_size = 31361584 2024/11/24-05:36:57.195157 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-05:36:57.196483 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:36:58.011160 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293744.sst): second pass time = 1.30's, 83.133'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.259'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.467'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 265.710'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26996.342'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.133'MB/sec zip pipeline throughput = 83.133'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.6933 dict = 1.00 all = 3.7997 } Zip/UnZip{ index = 0.0957 value = 0.2708 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2253.390781 GB avg = 0.199 KB (by entry num) total key len = 307.350579 GB avg = 0.027 KB total ukey len = 237.688739 GB avg = 0.015 KB total ukey num = 15.755361769 Billion total entry num = 11.296675717 Billion write speed all = 34.369693103 MB/sec (with seq num) write speed all = 33.156721630 MB/sec (without seq num) 2024/11/24-05:36:58.053657 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293744: 919300 keys, 35323442 bytes, temperature: kWarm 2024/11/24-05:36:58.053726 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397818053691, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293744, "file_size": 35323442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4297658290, "largest_seqno": 4407804745, "table_properties": {"data_size": 29371232, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397814, "oldest_key_time": 0, "file_creation_time": 1732397814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:36:58.093238 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst) /dev/shm/ToplingTemp/Topling-281184-HjQ8XZ 2024/11/24-05:36:59.098378 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst): first pass time = 1.01's, 133.536'MB/sec 2024/11/24-05:37:00.419688 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst): old prealloc_size = 147639500, real_size = 30447696 2024/11/24-05:37:00.420253 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931847 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0284 GB avg-val = 116.03 avg-zval = 30.52 2024/11/24-05:37:00.421040 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:37:01.244694 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293745.sst): second pass time = 1.32's, 81.907'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.892'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.008'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 266.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32496.036'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.907'MB/sec zip pipeline throughput = 81.907'MB/sec entries = 931847 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 30.52 usrkeys = 931847 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 30.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931847 bytes = 3261472 } seq expand size = 6522929 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.5123 value = 3.8017 dict = 1.00 all = 3.8836 } Zip/UnZip{ index = 0.0951 value = 0.2630 dict = 1.00 all = 0.2575 } ---------------------------- total value len = 2253.505430 GB avg = 0.199 KB (by entry num) total key len = 307.376671 GB avg = 0.027 KB total ukey len = 237.707376 GB avg = 0.015 KB total ukey num = 15.756293616 Billion total entry num = 11.297607564 Billion write speed all = 34.370090439 MB/sec (with seq num) write speed all = 33.157071555 MB/sec (without seq num) 2024/11/24-05:37:01.287198 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293745: 931847 keys, 34559926 bytes, temperature: kWarm 2024/11/24-05:37:01.287266 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397821287232, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293745, "file_size": 34559926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4461976370, "table_properties": {"data_size": 28441664, "index_size": 1772864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26091716, "raw_average_key_size": 28, "raw_value_size": 108126106, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931847, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397818, "oldest_key_time": 0, "file_creation_time": 1732397818, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:01.345349 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst) /dev/shm/ToplingTemp/Topling-281184-DuRXo0 2024/11/24-05:37:02.298758 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst): first pass time = 0.95's, 140.782'MB/sec 2024/11/24-05:37:03.581671 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst): old prealloc_size = 147639500, real_size = 31351360 2024/11/24-05:37:03.582233 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.94 2024/11/24-05:37:03.583139 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:37:04.399023 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293746.sst): second pass time = 1.28's, 84.622'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.207'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.340'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 265.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 29106.688'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.622'MB/sec zip pipeline throughput = 84.622'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.94 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.6946 dict = 1.00 all = 3.8008 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2253.620342 GB avg = 0.199 KB (by entry num) total key len = 307.402412 GB avg = 0.027 KB total ukey len = 237.725762 GB avg = 0.015 KB total ukey num = 15.757212916 Billion total entry num = 11.298526864 Billion write speed all = 34.370523106 MB/sec (with seq num) write speed all = 33.157456871 MB/sec (without seq num) 2024/11/24-05:37:04.442598 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293746: 919300 keys, 35313202 bytes, temperature: kWarm 2024/11/24-05:37:04.442668 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397824442633, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293746, "file_size": 35313202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269251946, "largest_seqno": 4377418228, "table_properties": {"data_size": 29360944, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397821, "oldest_key_time": 0, "file_creation_time": 1732397821, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:04.482222 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst) /dev/shm/ToplingTemp/Topling-281184-jVJBW2 2024/11/24-05:37:05.473841 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst): first pass time = 0.99's, 135.358'MB/sec 2024/11/24-05:37:06.780425 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst): old prealloc_size = 147639500, real_size = 30840576 2024/11/24-05:37:06.780962 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925680 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.99 avg-zval = 31.14 2024/11/24-05:37:06.781736 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:37:07.597650 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293747.sst): second pass time = 1.31's, 82.970'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.606'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3322.288'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 270.070'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31473.001'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.970'MB/sec zip pipeline throughput = 82.970'MB/sec entries = 925680 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.99 avg-zval = 31.14 usrkeys = 925680 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.99 avg-zval = 31.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925680 bytes = 3124192 } seq expand size = 6479760 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3794 value = 3.7571 dict = 1.00 all = 3.8549 } Zip/UnZip{ index = 0.0963 value = 0.2662 dict = 1.00 all = 0.2594 } ---------------------------- total value len = 2253.735121 GB avg = 0.199 KB (by entry num) total key len = 307.428331 GB avg = 0.027 KB total ukey len = 237.744276 GB avg = 0.015 KB total ukey num = 15.758138596 Billion total entry num = 11.299452544 Billion write speed all = 34.370935888 MB/sec (with seq num) write speed all = 33.157822344 MB/sec (without seq num) 2024/11/24-05:37:07.642025 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293747: 925680 keys, 34817890 bytes, temperature: kWarm 2024/11/24-05:37:07.642097 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397827642061, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293747, "file_size": 34817890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4377418229, "largest_seqno": 4461976097, "table_properties": {"data_size": 28825264, "index_size": 1783680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919040, "raw_average_key_size": 28, "raw_value_size": 108298828, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397824, "oldest_key_time": 0, "file_creation_time": 1732397824, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:07.678147 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst) /dev/shm/ToplingTemp/Topling-281184-gbiBv0 2024/11/24-05:37:08.503545 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst): first pass time = 0.83's, 162.616'MB/sec 2024/11/24-05:37:09.846056 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst): old prealloc_size = 147639500, real_size = 32937408 2024/11/24-05:37:09.846584 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925620 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.00 avg-zval = 33.43 2024/11/24-05:37:09.848008 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:37:10.703714 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293748.sst): second pass time = 1.34's, 80.743'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.179'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.145'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 274.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36904.718'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.743'MB/sec zip pipeline throughput = 80.743'MB/sec entries = 925620 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 33.43 usrkeys = 925620 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 33.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925620 bytes = 3239680 } seq expand size = 6479340 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5131 value = 3.4998 dict = 1.00 all = 3.6244 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2253.849901 GB avg = 0.199 KB (by entry num) total key len = 307.454248 GB avg = 0.027 KB total ukey len = 237.762788 GB avg = 0.015 KB total ukey num = 15.759064216 Billion total entry num = 11.300378164 Billion write speed all = 34.371391331 MB/sec (with seq num) write speed all = 33.158228981 MB/sec (without seq num) 2024/11/24-05:37:10.750470 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293748: 925620 keys, 37031806 bytes, temperature: kWarm 2024/11/24-05:37:10.750542 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397830750507, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293748, "file_size": 37031806, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4461975468, "table_properties": {"data_size": 30944944, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917360, "raw_average_key_size": 28, "raw_value_size": 108300492, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925620, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397827, "oldest_key_time": 0, "file_creation_time": 1732397827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:10.787527 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst) /dev/shm/ToplingTemp/Topling-281184-agVHl3 2024/11/24-05:37:11.629241 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst): first pass time = 0.84's, 159.465'MB/sec 2024/11/24-05:37:12.936603 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst): old prealloc_size = 147639500, real_size = 31299216 2024/11/24-05:37:12.937170 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.88 2024/11/24-05:37:12.937986 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:37:13.756450 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293749.sst): second pass time = 1.31's, 83.050'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.327'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 270.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32741.181'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.050'MB/sec zip pipeline throughput = 83.050'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.88 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.7012 dict = 1.00 all = 3.8064 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2253.964813 GB avg = 0.199 KB (by entry num) total key len = 307.479988 GB avg = 0.027 KB total ukey len = 237.781174 GB avg = 0.015 KB total ukey num = 15.759983516 Billion total entry num = 11.301297464 Billion write speed all = 34.371870736 MB/sec (with seq num) write speed all = 33.158659394 MB/sec (without seq num) 2024/11/24-05:37:13.799580 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293749: 919300 keys, 35261042 bytes, temperature: kWarm 2024/11/24-05:37:13.799650 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397833799614, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293749, "file_size": 35261042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4346507542, "largest_seqno": 4458618345, "table_properties": {"data_size": 29308800, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397830, "oldest_key_time": 0, "file_creation_time": 1732397830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:13.837054 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst) /dev/shm/ToplingTemp/Topling-281184-2LZLe0 2024/11/24-05:37:13.918196 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst): first pass time = 0.08's, 222.884'MB/sec 2024/11/24-05:37:14.087267 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst): old prealloc_size = 147639500, real_size = 5070288 2024/11/24-05:37:14.087404 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 126526 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0145 GB zip-val = 0.0048 GB avg-val = 114.87 avg-zval = 37.90 2024/11/24-05:37:14.087545 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/24-05:37:14.198772 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00/293750.sst): second pass time = 0.17's, 86.316'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 50.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2371.040'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 0.337'MB/sec write SST data time = 0.02's, 321.412'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.152'MB, throughput = 5580.559'MB/sec zip my value time = 0.17's, unzip length = 0.015'GB zip my value throughput = 86.316'MB/sec zip pipeline throughput = 86.316'MB/sec entries = 126526 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.87 avg-zval = 37.90 usrkeys = 126526 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.87 avg-zval = 37.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 126526 bytes = 411232 } seq expand size = 885682 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0025 GB value = 0.0145 GB dict = 0.15 MB all = 0.0181 GB } __ZipSize{ index = 0.0002 GB value = 0.0048 GB dict = 0.15 MB all = 0.0056 GB } UnZip/Zip{ index = 10.3914 value = 3.0312 dict = 1.00 all = 3.2260 } Zip/UnZip{ index = 0.0962 value = 0.3299 dict = 1.00 all = 0.3100 } ---------------------------- total value len = 2253.980234 GB avg = 0.199 KB (by entry num) total key len = 307.483531 GB avg = 0.027 KB total ukey len = 237.783705 GB avg = 0.015 KB total ukey num = 15.760110042 Billion total entry num = 11.301423990 Billion write speed all = 34.371921181 MB/sec (with seq num) write speed all = 33.158703458 MB/sec (without seq num) 2024/11/24-05:37:14.212756 139957215163968 [compaction_job.cc:1948] [default] [JOB 42285] Generated table #293750: 126526 keys, 5603645 bytes, temperature: kWarm 2024/11/24-05:37:14.212807 139957215163968 EVENT_LOG_v1 {"time_micros": 1732397834212780, "cf_name": "default", "job": 42285, "event": "table_file_creation", "file_number": 293750, "file_size": 5603645, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4411071743, "largest_seqno": 4461976108, "table_properties": {"data_size": 4794944, "index_size": 243520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3542728, "raw_average_key_size": 28, "raw_value_size": 14534540, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 126526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732397833, "oldest_key_time": 0, "file_creation_time": 1732397833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 293750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:37:14.225752 139957215163968 [compaction_job.cc:703] job-42285: subcompact[0], size: 0.818597 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42285/att-00 [293727,293728,293729,293730,293731,293732,293733,293734,293735,293736,293737,293738,293739,293740,293741,293742,293743,293744,293745,293746,293747,293748,293749,293750] 2024/11/24-05:37:14.356850 139957215163968 (Original Log Time 2024/11/24-05:35:59.751473) EVENT_LOG_v1 {"time_micros": 1732397759751432, "job": 42285, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [293673], "files_L3": [293724, 293629, 293630, 293631, 293632, 293633, 293634, 293635, 293636, 293637, 293638, 293639, 293640, 293600, 293601, 293602, 293603, 293604, 293605, 293606, 293607, 293608, 293609, 293690], "score": 0.899525, "input_data_size": 844516298, "oldest_snapshot_seqno": 4462363750} 2024/11/24-05:37:14.356854 139957215163968 (Original Log Time 2024/11/24-05:37:14.355231) [compaction_job.cc:2022] [default] [JOB 42285] Compacted 1@2 + 24@3 files to L3 => 818597425 bytes 2024/11/24-05:37:14.356855 139957215163968 (Original Log Time 2024/11/24-05:37:14.356782) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 11.3 rd, 11.0 wr, level 3, files in(1, 24) out(24 +0 blob) MB in(27.1, 778.3 +0.0 blob) out(780.7 +0.0 blob), read-write-amplify(58.5) write-amplify(28.8) OK, records in: 21545581, records dropped: 96702 output_compression: Snappy 2024/11/24-05:37:14.356858 139957215163968 (Original Log Time 2024/11/24-05:37:14.356826) EVENT_LOG_v1 {"time_micros": 1732397834356805, "cf": "default", "job": 42285, "event": "compaction_finished", "compaction_time_micros": 74474285, "compaction_time_cpu_micros": 32027247, "output_level": 3, "num_output_files": 24, "total_output_size": 818597425, "num_input_records": 21545581, "num_output_records": 21448879, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 560723619, "file_range_sync_nanos": 0, "file_fsync_nanos": 17340, "file_prepare_write_nanos": 63062, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/24-05:37:14.498820 139957215163968 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732397759,"executesMs":74725,"compactionJobId":42285,"attempt":0,"compactionInputRawBytes":3118818450,"compactionInputZipBytes":844516298,"compactionOutputRawBytes":0,"compactionOutputZipBytes":818597425,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-4bde791a426209de8012db98f8a5bc55-dc6e5987f256e1e8-00"}}, response = 2024/11/24-05:37:14.498903 139957215163968 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42285/att-00: olev 3, work 74.726 s, result 5.981 ms, install 1.644 ms, input{raw 2.905 GiB zip 805.394 MiB}