2024/11/24-00:34:03.630159 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00: fork to child time = 0.008492 sec 2024/11/24-00:34:03.630968 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00 2024/11/24-00:34:03.637275 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00 2024/11/24-00:34:03.638247 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:34:03.638518 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:34:03.708586 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38652/att-00/MANIFEST-206347 2024/11/24-00:34:03.725100 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38652/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 285929, last_sequence is 4351276230, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 285814 2024/11/24-00:34:03.725109 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:34:03.725261 139957634799168 [version_set.cc:5721] Creating manifest 285929 2024/11/24-00:34:03.761200 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:34:03.780349 139957634799168 [compaction_job.cc:2437] [default] [JOB 38652] Compacting 1@2 + 13@3 files to L3, score 0.48, subcompactions 1 : 1 2024/11/24-00:34:03.780364 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 28957 Base level 2, inputs: [285926(27MB)], [285882(35MB) 285883(31MB) 285884(34MB) 285885(34MB) 285886(34MB) 285887(33MB) 285888(33MB) 285889(34MB) 285890(35MB) 285891(35MB) 285892(35MB) 285893(6322KB) 285863(35MB)] 2024/11/24-00:34:03.787796 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:34:03.787825 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:34:03.787844 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:34:03.787850 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:34:03.787860 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:34:04.344778 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst) /dev/shm/ToplingTemp/Topling-227786-7lTbNi 2024/11/24-00:34:05.109580 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst): first pass time = 0.76's, 175.503'MB/sec 2024/11/24-00:34:06.796028 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst): old prealloc_size = 147639500, real_size = 32762864 2024/11/24-00:34:06.796619 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929175 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.45 avg-zval = 33.11 2024/11/24-00:34:06.797460 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:08.377484 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285930.sst): second pass time = 1.69's, 64.191'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 23.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3053.141'MB/sec (index lex order gen) rebuild zvType time = 1.34's, 0.173'MB/sec write SST data time = 0.24's, 154.244'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34186.161'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.191'MB/sec zip pipeline throughput = 64.191'MB/sec entries = 929175 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.45 avg-zval = 33.11 usrkeys = 929175 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.45 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929175 bytes = 3135984 } seq expand size = 6504225 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5137 value = 3.5173 dict = 1.00 all = 3.6517 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 8.963325 GB avg = 0.168 KB (by entry num) total key len = 1.466838 GB avg = 0.028 KB total ukey len = 1.016462 GB avg = 0.015 KB total ukey num = 0.066479610 Billion total entry num = 0.053287339 Billion write speed all = 0.185141522 MB/sec (with seq num) write speed all = 0.177574469 MB/sec (without seq num) 2024/11/24-00:34:08.425731 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285930: 929175 keys, 36755164 bytes, temperature: kWarm 2024/11/24-00:34:08.425859 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379648425778, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285930, "file_size": 36755164, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4351259275, "table_properties": {"data_size": 30762848, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016900, "raw_average_key_size": 28, "raw_value_size": 108200910, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379644, "oldest_key_time": 0, "file_creation_time": 1732379644, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285930, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:08.496873 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst) /dev/shm/ToplingTemp/Topling-227786-o9R1di 2024/11/24-00:34:09.501239 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst): first pass time = 1.00's, 133.641'MB/sec 2024/11/24-00:34:11.192456 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst): old prealloc_size = 147639500, real_size = 29138096 2024/11/24-00:34:11.193151 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937422 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0271 GB avg-val = 115.18 avg-zval = 28.92 2024/11/24-00:34:11.194011 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:12.499483 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285931.sst): second pass time = 1.69's, 63.873'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 25.362'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2671.242'MB/sec (index lex order gen) rebuild zvType time = 1.03's, 0.227'MB/sec write SST data time = 0.27's, 121.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33573.330'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.873'MB/sec zip pipeline throughput = 63.873'MB/sec entries = 937422 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.18 avg-zval = 28.92 usrkeys = 937422 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.18 avg-zval = 28.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937422 bytes = 3163808 } seq expand size = 6561954 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0271 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.4436 value = 3.9829 dict = 1.00 all = 4.0490 } Zip/UnZip{ index = 0.0958 value = 0.2511 dict = 1.00 all = 0.2470 } ---------------------------- total value len = 9.077857 GB avg = 0.167 KB (by entry num) total key len = 1.493086 GB avg = 0.028 KB total ukey len = 1.035210 GB avg = 0.015 KB total ukey num = 0.067417032 Billion total entry num = 0.054224761 Billion write speed all = 0.187626717 MB/sec (with seq num) write speed all = 0.179927110 MB/sec (without seq num) 2024/11/24-00:34:12.544337 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285931: 937422 keys, 33148698 bytes, temperature: kWarm 2024/11/24-00:34:12.544422 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379652544374, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285931, "file_size": 33148698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4351259367, "table_properties": {"data_size": 27108384, "index_size": 1795200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26247816, "raw_average_key_size": 28, "raw_value_size": 107970012, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937422, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379648, "oldest_key_time": 0, "file_creation_time": 1732379648, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285931, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:12.614687 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst) /dev/shm/ToplingTemp/Topling-227786-C4tdbk 2024/11/24-00:34:13.690889 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst): first pass time = 1.08's, 124.720'MB/sec 2024/11/24-00:34:15.395339 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst): old prealloc_size = 147639500, real_size = 31980224 2024/11/24-00:34:15.396032 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929373 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.42 avg-zval = 32.26 2024/11/24-00:34:15.396822 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:16.258011 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285932.sst): second pass time = 1.70's, 63.518'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2637.704'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.14's, 252.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30296.207'MB/sec zip my value time = 1.70's, unzip length = 0.108'GB zip my value throughput = 63.518'MB/sec zip pipeline throughput = 65.528'MB/sec entries = 929373 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 32.26 usrkeys = 929373 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 32.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929373 bytes = 3136656 } seq expand size = 6505611 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5117 value = 3.6090 dict = 1.00 all = 3.7314 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 9.192558 GB avg = 0.167 KB (by entry num) total key len = 1.519108 GB avg = 0.028 KB total ukey len = 1.053797 GB avg = 0.015 KB total ukey num = 0.068346405 Billion total entry num = 0.055154134 Billion write speed all = 0.190111776 MB/sec (with seq num) write speed all = 0.182280725 MB/sec (without seq num) 2024/11/24-00:34:16.303470 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285932: 929373 keys, 35969444 bytes, temperature: kWarm 2024/11/24-00:34:16.303540 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379656303504, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285932, "file_size": 35969444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4351259383, "table_properties": {"data_size": 29979440, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022444, "raw_average_key_size": 28, "raw_value_size": 108195414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929373, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379652, "oldest_key_time": 0, "file_creation_time": 1732379652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285932, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:16.355504 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst) /dev/shm/ToplingTemp/Topling-227786-jRwlTi 2024/11/24-00:34:17.271955 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst): first pass time = 0.92's, 146.460'MB/sec 2024/11/24-00:34:18.756658 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst): old prealloc_size = 147639500, real_size = 32015296 2024/11/24-00:34:18.757232 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935150 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0300 GB avg-val = 115.53 avg-zval = 32.08 2024/11/24-00:34:18.757946 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:19.623007 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285933.sst): second pass time = 1.48's, 72.824'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.581'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.432'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 278.072'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36104.234'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.824'MB/sec zip pipeline throughput = 94.011'MB/sec entries = 935150 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.53 avg-zval = 32.08 usrkeys = 935150 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.53 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935150 bytes = 3156144 } seq expand size = 6546050 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5135 value = 3.6008 dict = 1.00 all = 3.7261 } Zip/UnZip{ index = 0.0951 value = 0.2777 dict = 1.00 all = 0.2684 } ---------------------------- total value len = 9.307138 GB avg = 0.166 KB (by entry num) total key len = 1.545292 GB avg = 0.028 KB total ukey len = 1.072500 GB avg = 0.015 KB total ukey num = 0.069281555 Billion total entry num = 0.056089284 Billion write speed all = 0.192598565 MB/sec (with seq num) write speed all = 0.184635212 MB/sec (without seq num) 2024/11/24-00:34:19.668512 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285933: 935150 keys, 36021432 bytes, temperature: kWarm 2024/11/24-00:34:19.668584 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379659668549, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285933, "file_size": 36021432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4351259651, "table_properties": {"data_size": 30002352, "index_size": 1778944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26184200, "raw_average_key_size": 28, "raw_value_size": 108033572, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935150, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379656, "oldest_key_time": 0, "file_creation_time": 1732379656, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285933, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:19.705371 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst) /dev/shm/ToplingTemp/Topling-227786-HrGINh 2024/11/24-00:34:20.615874 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst): first pass time = 0.91's, 147.416'MB/sec 2024/11/24-00:34:22.150511 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst): old prealloc_size = 147639500, real_size = 31631616 2024/11/24-00:34:22.151055 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931889 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.03 avg-zval = 31.79 2024/11/24-00:34:22.151796 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:22.991772 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285934.sst): second pass time = 1.53's, 70.514'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.351'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3406.404'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 275.321'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33441.811'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.514'MB/sec zip pipeline throughput = 104.357'MB/sec entries = 931889 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 31.79 usrkeys = 931889 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931889 bytes = 3145136 } seq expand size = 6523223 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5139 value = 3.6497 dict = 1.00 all = 3.7672 } Zip/UnZip{ index = 0.0951 value = 0.2740 dict = 1.00 all = 0.2654 } ---------------------------- total value len = 9.421786 GB avg = 0.165 KB (by entry num) total key len = 1.571385 GB avg = 0.028 KB total ukey len = 1.091138 GB avg = 0.016 KB total ukey num = 0.070213444 Billion total entry num = 0.057021173 Billion write speed all = 0.195084640 MB/sec (with seq num) write speed all = 0.186989465 MB/sec (without seq num) 2024/11/24-00:34:23.036649 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285934: 931889 keys, 35627748 bytes, temperature: kWarm 2024/11/24-00:34:23.036721 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379663036683, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285934, "file_size": 35627748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4351259402, "table_properties": {"data_size": 29625776, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092892, "raw_average_key_size": 28, "raw_value_size": 108124978, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931889, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379659, "oldest_key_time": 0, "file_creation_time": 1732379659, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285934, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:23.072369 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst) /dev/shm/ToplingTemp/Topling-227786-w9DV2i 2024/11/24-00:34:23.968473 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst): first pass time = 0.90's, 149.786'MB/sec 2024/11/24-00:34:25.490497 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst): old prealloc_size = 147639500, real_size = 30377552 2024/11/24-00:34:25.491074 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928722 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.52 avg-zval = 30.54 2024/11/24-00:34:25.491847 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:26.319222 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285935.sst): second pass time = 1.52's, 71.128'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.467'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.926'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 262.383'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34165.802'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.128'MB/sec zip pipeline throughput = 114.667'MB/sec entries = 928722 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 30.54 usrkeys = 928722 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 30.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928722 bytes = 3134448 } seq expand size = 6501054 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4447 value = 3.8148 dict = 1.00 all = 3.9058 } Zip/UnZip{ index = 0.0957 value = 0.2621 dict = 1.00 all = 0.2560 } ---------------------------- total value len = 9.536500 GB avg = 0.165 KB (by entry num) total key len = 1.597390 GB avg = 0.028 KB total ukey len = 1.109713 GB avg = 0.016 KB total ukey num = 0.071142166 Billion total entry num = 0.057949895 Billion write speed all = 0.197570168 MB/sec (with seq num) write speed all = 0.189343631 MB/sec (without seq num) 2024/11/24-00:34:26.368468 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285935: 928722 keys, 34363568 bytes, temperature: kWarm 2024/11/24-00:34:26.368534 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379666368499, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285935, "file_size": 34363568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4281820259, "largest_seqno": 4351259414, "table_properties": {"data_size": 28366848, "index_size": 1778368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004216, "raw_average_key_size": 28, "raw_value_size": 108213576, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928722, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379663, "oldest_key_time": 0, "file_creation_time": 1732379663, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285935, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:26.409261 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst) /dev/shm/ToplingTemp/Topling-227786-rPyMph 2024/11/24-00:34:27.316153 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst): first pass time = 0.91's, 148.004'MB/sec 2024/11/24-00:34:28.626760 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst): old prealloc_size = 147639500, real_size = 31275872 2024/11/24-00:34:28.627375 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924499 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0293 GB avg-val = 117.18 avg-zval = 31.68 2024/11/24-00:34:28.628138 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:29.448031 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285936.sst): second pass time = 1.31's, 82.707'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2982.092'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 271.879'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32035.897'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.707'MB/sec zip pipeline throughput = 82.707'MB/sec entries = 924499 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 31.68 usrkeys = 924499 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 31.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924499 bytes = 3120192 } seq expand size = 6471493 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5137 value = 3.6991 dict = 1.00 all = 3.8073 } Zip/UnZip{ index = 0.0951 value = 0.2703 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 9.651304 GB avg = 0.164 KB (by entry num) total key len = 1.623276 GB avg = 0.028 KB total ukey len = 1.128203 GB avg = 0.016 KB total ukey num = 0.072066665 Billion total entry num = 0.058874394 Billion write speed all = 0.200055585 MB/sec (with seq num) write speed all = 0.191698270 MB/sec (without seq num) 2024/11/24-00:34:29.491517 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285936: 924499 keys, 35252396 bytes, temperature: kWarm 2024/11/24-00:34:29.491588 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379669491553, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285936, "file_size": 35252396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4351259916, "table_properties": {"data_size": 29285904, "index_size": 1758656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885972, "raw_average_key_size": 28, "raw_value_size": 108331882, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924499, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379666, "oldest_key_time": 0, "file_creation_time": 1732379666, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285936, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:29.531688 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst) /dev/shm/ToplingTemp/Topling-227786-FeVHTj 2024/11/24-00:34:30.558857 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst): first pass time = 1.03's, 130.673'MB/sec 2024/11/24-00:34:31.900236 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst): old prealloc_size = 147639500, real_size = 32758192 2024/11/24-00:34:31.900811 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924506 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.18 avg-zval = 33.28 2024/11/24-00:34:31.901626 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:32.739237 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285937.sst): second pass time = 1.34's, 80.815'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.785'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.452'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 285.837'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28393.152'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.815'MB/sec zip pipeline throughput = 80.815'MB/sec entries = 924506 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.28 usrkeys = 924506 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.18 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924506 bytes = 3120224 } seq expand size = 6471542 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5122 value = 3.5209 dict = 1.00 all = 3.6537 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 9.766107 GB avg = 0.163 KB (by entry num) total key len = 1.649162 GB avg = 0.028 KB total ukey len = 1.146693 GB avg = 0.016 KB total ukey num = 0.072991171 Billion total entry num = 0.059798900 Billion write speed all = 0.202540140 MB/sec (with seq num) write speed all = 0.194052086 MB/sec (without seq num) 2024/11/24-00:34:32.786265 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285937: 924506 keys, 36734404 bytes, temperature: kWarm 2024/11/24-00:34:32.786342 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379672786303, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285937, "file_size": 36734404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4351259446, "table_properties": {"data_size": 30767968, "index_size": 1758912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25886168, "raw_average_key_size": 28, "raw_value_size": 108331588, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924506, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379669, "oldest_key_time": 0, "file_creation_time": 1732379669, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285937, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:32.825299 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst) /dev/shm/ToplingTemp/Topling-227786-CRzCli 2024/11/24-00:34:33.846131 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst): first pass time = 1.02's, 131.486'MB/sec 2024/11/24-00:34:35.170595 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst): old prealloc_size = 147639500, real_size = 32903472 2024/11/24-00:34:35.171151 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928569 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.54 avg-zval = 33.28 2024/11/24-00:34:35.171882 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:36.029583 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285938.sst): second pass time = 1.32's, 81.781'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.947'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3319.933'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 284.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 26748.550'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.781'MB/sec zip pipeline throughput = 81.781'MB/sec entries = 928569 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 33.28 usrkeys = 928569 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928569 bytes = 3133936 } seq expand size = 6499983 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5137 value = 3.5017 dict = 1.00 all = 3.6383 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 9.880825 GB avg = 0.163 KB (by entry num) total key len = 1.675162 GB avg = 0.028 KB total ukey len = 1.165264 GB avg = 0.016 KB total ukey num = 0.073919740 Billion total entry num = 0.060727469 Billion write speed all = 0.205024916 MB/sec (with seq num) write speed all = 0.196405560 MB/sec (without seq num) 2024/11/24-00:34:36.076440 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285938: 928569 keys, 36890392 bytes, temperature: kWarm 2024/11/24-00:34:36.076510 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379676076476, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285938, "file_size": 36890392, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4351259679, "table_properties": {"data_size": 30904736, "index_size": 1766400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999932, "raw_average_key_size": 28, "raw_value_size": 108217934, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928569, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379672, "oldest_key_time": 0, "file_creation_time": 1732379672, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285938, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:36.117488 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst) /dev/shm/ToplingTemp/Topling-227786-yVvBnk 2024/11/24-00:34:36.985564 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst): first pass time = 0.87's, 154.622'MB/sec 2024/11/24-00:34:38.331615 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst): old prealloc_size = 147639500, real_size = 33075792 2024/11/24-00:34:38.332242 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929174 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.45 avg-zval = 33.44 2024/11/24-00:34:38.333020 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:39.188539 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285939.sst): second pass time = 1.34's, 80.468'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.255'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2918.708'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 294.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34167.569'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.468'MB/sec zip pipeline throughput = 80.468'MB/sec entries = 929174 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.45 avg-zval = 33.44 usrkeys = 929174 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.45 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929174 bytes = 3135984 } seq expand size = 6504218 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5137 value = 3.4818 dict = 1.00 all = 3.6212 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 9.995530 GB avg = 0.162 KB (by entry num) total key len = 1.701178 GB avg = 0.028 KB total ukey len = 1.183848 GB avg = 0.016 KB total ukey num = 0.074848914 Billion total entry num = 0.061656643 Billion write speed all = 0.207509960 MB/sec (with seq num) write speed all = 0.198759212 MB/sec (without seq num) 2024/11/24-00:34:39.233647 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285939: 929174 keys, 37064752 bytes, temperature: kWarm 2024/11/24-00:34:39.233718 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379679233681, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285939, "file_size": 37064752, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4351259623, "table_properties": {"data_size": 31075776, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016872, "raw_average_key_size": 28, "raw_value_size": 108200968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929174, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379676, "oldest_key_time": 0, "file_creation_time": 1732379676, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285939, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:39.271919 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst) /dev/shm/ToplingTemp/Topling-227786-Nhf6dl 2024/11/24-00:34:40.200404 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst): first pass time = 0.93's, 144.561'MB/sec 2024/11/24-00:34:41.535097 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst): old prealloc_size = 147639500, real_size = 32712144 2024/11/24-00:34:41.535631 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928937 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.49 avg-zval = 33.06 2024/11/24-00:34:41.536454 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:42.379198 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285940.sst): second pass time = 1.33's, 81.158'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.480'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3546.509'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 292.635'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34126.585'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.158'MB/sec zip pipeline throughput = 81.158'MB/sec entries = 928937 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.06 usrkeys = 928937 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928937 bytes = 3135184 } seq expand size = 6502559 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5121 value = 3.5233 dict = 1.00 all = 3.6571 } Zip/UnZip{ index = 0.0951 value = 0.2838 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 10.110240 GB avg = 0.162 KB (by entry num) total key len = 1.727189 GB avg = 0.028 KB total ukey len = 1.202426 GB avg = 0.016 KB total ukey num = 0.075777851 Billion total entry num = 0.062585580 Billion write speed all = 0.209994576 MB/sec (with seq num) write speed all = 0.201112490 MB/sec (without seq num) 2024/11/24-00:34:42.425326 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285940: 928937 keys, 36700402 bytes, temperature: kWarm 2024/11/24-00:34:42.425395 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379682425359, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285940, "file_size": 36700402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4351259561, "table_properties": {"data_size": 30712384, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26010236, "raw_average_key_size": 28, "raw_value_size": 108207554, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928937, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379679, "oldest_key_time": 0, "file_creation_time": 1732379679, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285940, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:42.472276 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst) /dev/shm/ToplingTemp/Topling-227786-KvyCEh 2024/11/24-00:34:43.594505 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst): first pass time = 1.12's, 119.604'MB/sec 2024/11/24-00:34:44.910657 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst): old prealloc_size = 147639500, real_size = 31859744 2024/11/24-00:34:44.911287 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928585 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.54 avg-zval = 32.16 2024/11/24-00:34:44.912039 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:34:45.745411 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285941.sst): second pass time = 1.32's, 82.273'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.944'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2909.153'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 280.347'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29900.690'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.273'MB/sec zip pipeline throughput = 82.273'MB/sec entries = 928585 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 32.16 usrkeys = 928585 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928585 bytes = 3133984 } seq expand size = 6500095 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5127 value = 3.6241 dict = 1.00 all = 3.7442 } Zip/UnZip{ index = 0.0951 value = 0.2759 dict = 1.00 all = 0.2671 } ---------------------------- total value len = 10.224958 GB avg = 0.161 KB (by entry num) total key len = 1.753189 GB avg = 0.028 KB total ukey len = 1.220998 GB avg = 0.016 KB total ukey num = 0.076706436 Billion total entry num = 0.063514165 Billion write speed all = 0.212478206 MB/sec (with seq num) write speed all = 0.203464874 MB/sec (without seq num) 2024/11/24-00:34:45.789651 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285941: 928585 keys, 35847060 bytes, temperature: kWarm 2024/11/24-00:34:45.789723 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379685789687, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285941, "file_size": 35847060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4351044544, "table_properties": {"data_size": 29860816, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000380, "raw_average_key_size": 28, "raw_value_size": 108217370, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928585, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379682, "oldest_key_time": 0, "file_creation_time": 1732379682, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285941, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:45.826958 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst) /dev/shm/ToplingTemp/Topling-227786-SzISOk 2024/11/24-00:34:45.986146 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst): first pass time = 0.16's, 246.833'MB/sec 2024/11/24-00:34:46.364112 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst): old prealloc_size = 147639500, real_size = 10819072 2024/11/24-00:34:46.364334 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 269041 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0317 GB zip-val = 0.0102 GB avg-val = 118.00 avg-zval = 38.03 2024/11/24-00:34:46.364573 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.039 GB 2024/11/24-00:34:46.606932 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00/285942.sst): second pass time = 0.38's, 84.266'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 49.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2799.093'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.328'MB/sec write SST data time = 0.04's, 315.985'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.324'MB, throughput = 8468.693'MB/sec zip my value time = 0.38's, unzip length = 0.032'GB zip my value throughput = 84.266'MB/sec zip pipeline throughput = 84.266'MB/sec entries = 269041 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 38.03 usrkeys = 269041 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 38.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 269041 bytes = 840768 } seq expand size = 1883287 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0054 GB value = 0.0317 GB dict = 0.32 MB all = 0.0393 GB } __ZipSize{ index = 0.0005 GB value = 0.0102 GB dict = 0.32 MB all = 0.0119 GB } UnZip/Zip{ index = 10.3528 value = 3.1027 dict = 1.00 all = 3.2957 } Zip/UnZip{ index = 0.0966 value = 0.3223 dict = 1.00 all = 0.3034 } ---------------------------- total value len = 10.258588 GB avg = 0.161 KB (by entry num) total key len = 1.760722 GB avg = 0.028 KB total ukey len = 1.226379 GB avg = 0.016 KB total ukey num = 0.076975477 Billion total entry num = 0.063783206 Billion write speed all = 0.213205135 MB/sec (with seq num) write speed all = 0.204153762 MB/sec (without seq num) 2024/11/24-00:34:46.626313 139957634799168 [compaction_job.cc:1948] [default] [JOB 38652] Generated table #285942: 269041 keys, 11918383 bytes, temperature: kWarm 2024/11/24-00:34:46.626380 139957634799168 EVENT_LOG_v1 {"time_micros": 1732379686626346, "cf_name": "default", "job": 38652, "event": "table_file_creation", "file_number": 285942, "file_size": 11918383, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4279476740, "largest_seqno": 4310799174, "table_properties": {"data_size": 10231856, "index_size": 519744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7533148, "raw_average_key_size": 28, "raw_value_size": 31746838, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 269041, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379685, "oldest_key_time": 0, "file_creation_time": 1732379685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285942, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:34:46.641915 139957634799168 [compaction_job.cc:703] job-38652: subcompact[0], size: 0.442294 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38652/att-00 [285930,285931,285932,285933,285934,285935,285936,285937,285938,285939,285940,285941,285942] 2024/11/24-00:34:46.721217 139957634799168 (Original Log Time 2024/11/24-00:34:03.780420) EVENT_LOG_v1 {"time_micros": 1732379643780379, "job": 38652, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [285926], "files_L3": [285882, 285883, 285884, 285885, 285886, 285887, 285888, 285889, 285890, 285891, 285892, 285893, 285863], "score": 0.475173, "input_data_size": 466965095, "oldest_snapshot_seqno": 4351276088} 2024/11/24-00:34:46.721221 139957634799168 (Original Log Time 2024/11/24-00:34:46.720134) [compaction_job.cc:2022] [default] [JOB 38652] Compacted 1@2 + 13@3 files to L3 => 442293843 bytes 2024/11/24-00:34:46.721222 139957634799168 (Original Log Time 2024/11/24-00:34:46.721155) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.3 wr, level 3, files in(1, 13) out(13 +0 blob) MB in(27.1, 418.3 +0.0 blob) out(421.8 +0.0 blob), read-write-amplify(32.0) write-amplify(15.6) OK, records in: 11525719, records dropped: 100677 output_compression: Snappy 2024/11/24-00:34:46.721228 139957634799168 (Original Log Time 2024/11/24-00:34:46.721197) EVENT_LOG_v1 {"time_micros": 1732379686721177, "cf": "default", "job": 38652, "event": "compaction_finished", "compaction_time_micros": 42861502, "compaction_time_cpu_micros": 18448847, "output_level": 3, "num_output_files": 13, "total_output_size": 442293843, "num_input_records": 11525719, "num_output_records": 11425042, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 334488236, "file_range_sync_nanos": 0, "file_fsync_nanos": 10886, "file_prepare_write_nanos": 35008, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/24-00:34:46.874212 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732379643,"executesMs":43082,"compactionJobId":38652,"attempt":0,"compactionInputRawBytes":1664189906,"compactionInputZipBytes":466965095,"compactionOutputRawBytes":0,"compactionOutputZipBytes":442293843,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6110ffc9c64e6bebbc25657af10d418e-5a26e86bbfbe051a-00"}}, response = 2024/11/24-00:34:46.874286 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38652/att-00: olev 3, work 43.082 s, result 7.755 ms, install 1.117 ms, input{raw 1.550 GiB zip 445.333 MiB}