2024/11/22-20:29:26.307207 140025221817920 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00: fork to child time = 0.009542 sec 2024/11/22-20:29:26.308272 140025221817920 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00 2024/11/22-20:29:26.315985 140025221817920 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00 2024/11/22-20:29:26.317116 140025221817920 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-20:29:26.317383 140025221817920 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-20:29:26.365567 140025221817920 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-10232/att-00/MANIFEST-206347 2024/11/22-20:29:26.398845 140025221817920 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-10232/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 226141, last_sequence is 3540946725, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 226002 2024/11/22-20:29:26.398852 140025221817920 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-20:29:26.398943 140025221817920 [version_set.cc:5721] Creating manifest 226141 2024/11/22-20:29:26.437199 140025221817920 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-20:29:26.458612 140025221817920 [compaction_job.cc:2437] [default] [JOB 10232] Compacting 1@2 + 10@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-20:29:26.458631 140025221817920 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7736 Base level 2, inputs: [226131(16MB)], [226098(35MB) 226099(35MB) 226100(35MB) 226101(35MB) 226102(35MB) 226103(35MB) 226104(35MB) 226105(35MB) 226106(33MB) 226107(35MB)] 2024/11/22-20:29:26.469480 140025221817920 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:29:26.469505 140025221817920 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:29:26.469522 140025221817920 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:29:26.469527 140025221817920 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:29:26.469537 140025221817920 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:29:26.867967 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst) /dev/shm/ToplingTemp/Topling-279411-pTuLxY 2024/11/22-20:29:27.825232 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst): first pass time = 0.96's, 140.217'MB/sec 2024/11/22-20:29:30.703647 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst): old prealloc_size = 147639500, real_size = 33197712 2024/11/22-20:29:30.704241 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923875 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.28 avg-zval = 33.78 2024/11/22-20:29:30.705042 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:31.961567 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226142.sst): second pass time = 2.88's, 37.651'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.26's, 29.643'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3021.963'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.214'MB/sec write SST data time = 0.18's, 208.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31440.400'MB/sec zip my value time = 2.88's, unzip length = 0.108'GB zip my value throughput = 37.651'MB/sec zip pipeline throughput = 91.383'MB/sec entries = 923875 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 33.78 usrkeys = 923875 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 33.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923875 bytes = 3233584 } seq expand size = 6467125 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5135 value = 3.4717 dict = 1.00 all = 3.5995 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 0.114816 GB avg = 0.124 KB (by entry num) total key len = 0.025868 GB avg = 0.028 KB total ukey len = 0.018478 GB avg = 0.020 KB total ukey num = 0.000923875 Billion total entry num = 0.000923875 Billion write speed all = 0.003363347 MB/sec (with seq num) write speed all = 0.003186651 MB/sec (without seq num) 2024/11/22-20:29:31.995549 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226142: 923875 keys, 37288306 bytes, temperature: kWarm 2024/11/22-20:29:31.995631 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278571995579, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226142, "file_size": 37288306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3540923916, "table_properties": {"data_size": 31209024, "index_size": 1757504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868500, "raw_average_key_size": 28, "raw_value_size": 108349366, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923875, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278566, "oldest_key_time": 0, "file_creation_time": 1732278566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226142, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:32.033776 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst) /dev/shm/ToplingTemp/Topling-279411-H0vxWV 2024/11/22-20:29:33.113580 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst): first pass time = 1.08's, 124.304'MB/sec 2024/11/22-20:29:36.071545 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst): old prealloc_size = 147639500, real_size = 33459200 2024/11/22-20:29:36.072133 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923876 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.28 avg-zval = 34.05 2024/11/22-20:29:36.072904 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:37.178239 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226143.sst): second pass time = 2.96's, 36.645'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 33.930'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3104.333'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.19's, 198.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31438.074'MB/sec zip my value time = 2.96's, unzip length = 0.108'GB zip my value throughput = 36.645'MB/sec zip pipeline throughput = 114.848'MB/sec entries = 923876 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.28 avg-zval = 34.05 usrkeys = 923876 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.28 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923876 bytes = 3233584 } seq expand size = 6467132 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4447 value = 3.4442 dict = 1.00 all = 3.5744 } Zip/UnZip{ index = 0.0957 value = 0.2903 dict = 1.00 all = 0.2798 } ---------------------------- total value len = 0.229633 GB avg = 0.124 KB (by entry num) total key len = 0.051737 GB avg = 0.028 KB total ukey len = 0.036955 GB avg = 0.020 KB total ukey num = 0.001847751 Billion total entry num = 0.001847751 Billion write speed all = 0.006725855 MB/sec (with seq num) write speed all = 0.006372506 MB/sec (without seq num) 2024/11/22-20:29:37.212284 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226143: 923876 keys, 37549950 bytes, temperature: kWarm 2024/11/22-20:29:37.212341 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278577212310, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226143, "file_size": 37549950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3540923790, "table_properties": {"data_size": 31458928, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868528, "raw_average_key_size": 28, "raw_value_size": 108349308, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923876, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278572, "oldest_key_time": 0, "file_creation_time": 1732278572, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226143, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:37.247386 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst) /dev/shm/ToplingTemp/Topling-279411-TZVUeY 2024/11/22-20:29:38.573107 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst): first pass time = 1.33's, 101.244'MB/sec 2024/11/22-20:29:40.363326 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst): old prealloc_size = 147639500, real_size = 33594704 2024/11/22-20:29:40.363870 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923888 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.28 avg-zval = 34.21 2024/11/22-20:29:40.364602 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:41.501424 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226144.sst): second pass time = 1.79's, 60.543'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 33.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.265'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.239'MB/sec write SST data time = 0.17's, 219.261'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 38411.427'MB/sec zip my value time = 1.79's, unzip length = 0.108'GB zip my value throughput = 60.543'MB/sec zip pipeline throughput = 60.543'MB/sec entries = 923888 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 34.21 usrkeys = 923888 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 34.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923888 bytes = 3233616 } seq expand size = 6467216 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5144 value = 3.4281 dict = 1.00 all = 3.5615 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 0.344449 GB avg = 0.124 KB (by entry num) total key len = 0.077606 GB avg = 0.028 KB total ukey len = 0.055433 GB avg = 0.020 KB total ukey num = 0.002771639 Billion total entry num = 0.002771639 Billion write speed all = 0.010087741 MB/sec (with seq num) write speed all = 0.009557771 MB/sec (without seq num) 2024/11/22-20:29:41.552323 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226144: 923888 keys, 37685586 bytes, temperature: kWarm 2024/11/22-20:29:41.552375 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278581552346, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226144, "file_size": 37685586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3540923800, "table_properties": {"data_size": 31606144, "index_size": 1757376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868864, "raw_average_key_size": 28, "raw_value_size": 108348972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923888, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278577, "oldest_key_time": 0, "file_creation_time": 1732278577, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226144, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:41.601628 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst) /dev/shm/ToplingTemp/Topling-279411-cF47aW 2024/11/22-20:29:42.953021 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst): first pass time = 1.35's, 99.321'MB/sec 2024/11/22-20:29:44.557410 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst): old prealloc_size = 147639500, real_size = 33657808 2024/11/22-20:29:44.557977 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923932 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.27 avg-zval = 34.28 2024/11/22-20:29:44.558731 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:45.497401 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226145.sst): second pass time = 1.60's, 67.576'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.988'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 299.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33171.116'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.576'MB/sec zip pipeline throughput = 67.576'MB/sec entries = 923932 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 34.28 usrkeys = 923932 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.27 avg-zval = 34.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923932 bytes = 3233776 } seq expand size = 6467524 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5157 value = 3.4212 dict = 1.00 all = 3.5556 } Zip/UnZip{ index = 0.0951 value = 0.2923 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 0.459264 GB avg = 0.124 KB (by entry num) total key len = 0.103476 GB avg = 0.028 KB total ukey len = 0.073911 GB avg = 0.020 KB total ukey num = 0.003695571 Billion total entry num = 0.003695571 Billion write speed all = 0.013449046 MB/sec (with seq num) write speed all = 0.012742477 MB/sec (without seq num) 2024/11/22-20:29:45.530695 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226145: 923932 keys, 37748658 bytes, temperature: kWarm 2024/11/22-20:29:45.530756 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278585530726, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226145, "file_size": 37748658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774802, "largest_seqno": 3540923817, "table_properties": {"data_size": 31669376, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870096, "raw_average_key_size": 28, "raw_value_size": 108347764, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278581, "oldest_key_time": 0, "file_creation_time": 1732278581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226145, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:45.564952 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst) /dev/shm/ToplingTemp/Topling-279411-cLKS4Y 2024/11/22-20:29:46.531134 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst): first pass time = 0.97's, 138.922'MB/sec 2024/11/22-20:29:48.378118 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst): old prealloc_size = 147639500, real_size = 35236528 2024/11/22-20:29:48.378667 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.65 avg-zval = 35.80 2024/11/22-20:29:48.379098 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:49.178397 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226146.sst): second pass time = 1.85's, 58.621'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 47.104'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3394.500'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.345'MB/sec write SST data time = 0.13's, 294.650'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35468.295'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.621'MB/sec zip pipeline throughput = 58.621'MB/sec entries = 927890 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.65 avg-zval = 35.80 usrkeys = 927890 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.03 avg-zval = 35.80 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 447951 bytes = 1455856 } seq expand size = 3135657 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.3767 value = 3.2586 dict = 1.00 all = 3.5606 } Zip/UnZip{ index = 0.0964 value = 0.3069 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 0.570637 GB avg = 0.123 KB (by entry num) total key len = 0.129457 GB avg = 0.028 KB total ukey len = 0.092469 GB avg = 0.020 KB total ukey num = 0.004623461 Billion total entry num = 0.004623461 Billion write speed all = 0.016730211 MB/sec (with seq num) write speed all = 0.015846313 MB/sec (without seq num) 2024/11/22-20:29:49.211957 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226146: 927890 keys, 37695235 bytes, temperature: kWarm 2024/11/22-20:29:49.212018 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278589211986, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226146, "file_size": 37695235, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3540923835, "table_properties": {"data_size": 33215968, "index_size": 1788416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980920, "raw_average_key_size": 28, "raw_value_size": 108236824, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278585, "oldest_key_time": 0, "file_creation_time": 1732278585, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226146, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:49.251306 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst) /dev/shm/ToplingTemp/Topling-279411-xPiiaY 2024/11/22-20:29:50.385662 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst): first pass time = 1.13's, 118.325'MB/sec 2024/11/22-20:29:52.108693 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst): old prealloc_size = 147639500, real_size = 35428848 2024/11/22-20:29:52.109325 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080055 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.27 avg-zval = 30.64 2024/11/22-20:29:52.109661 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:52.770800 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226147.sst): second pass time = 1.72's, 60.386'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 55.830'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3387.932'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.516'MB/sec write SST data time = 0.14's, 266.861'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.025'MB, throughput = 33814.217'MB/sec zip my value time = 1.72's, unzip length = 0.104'GB zip my value throughput = 60.386'MB/sec zip pipeline throughput = 60.386'MB/sec entries = 1080055 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.27 avg-zval = 30.64 usrkeys = 1080055 avg-key = 20.00 avg-zkey = 1.91 avg-val = 97.38 avg-zval = 30.64 TagRS{ kinds = RS_Key0_TagN bytes = 168840 } TagArray{ size = 171591 bytes = 600576 } seq expand size = 1201137 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4475 value = 3.1421 dict = 1.00 all = 3.6310 } Zip/UnZip{ index = 0.0957 value = 0.3183 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 0.675814 GB avg = 0.118 KB (by entry num) total key len = 0.159698 GB avg = 0.028 KB total ukey len = 0.114070 GB avg = 0.020 KB total ukey num = 0.005703516 Billion total entry num = 0.005703516 Billion write speed all = 0.019964618 MB/sec (with seq num) write speed all = 0.018874332 MB/sec (without seq num) 2024/11/22-20:29:52.805127 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226147: 1080055 keys, 36964037 bytes, temperature: kWarm 2024/11/22-20:29:52.805191 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278592805158, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226147, "file_size": 36964037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3414458260, "table_properties": {"data_size": 33091040, "index_size": 2067584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30241540, "raw_average_key_size": 28, "raw_value_size": 103976311, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080055, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278589, "oldest_key_time": 0, "file_creation_time": 1732278589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226147, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:52.845356 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst) /dev/shm/ToplingTemp/Topling-279411-jvBmMV 2024/11/22-20:29:54.210337 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst): first pass time = 1.36's, 98.332'MB/sec 2024/11/22-20:29:55.578365 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst): old prealloc_size = 147639500, real_size = 32125152 2024/11/22-20:29:55.578957 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923796 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0301 GB avg-val = 117.29 avg-zval = 32.62 2024/11/22-20:29:55.579714 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:56.453595 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226148.sst): second pass time = 1.37's, 79.243'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.640'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 286.780'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35332.115'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.243'MB/sec zip pipeline throughput = 79.243'MB/sec entries = 923796 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 32.62 usrkeys = 923796 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 32.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923796 bytes = 3233296 } seq expand size = 6466572 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5122 value = 3.5954 dict = 1.00 all = 3.7060 } Zip/UnZip{ index = 0.0951 value = 0.2781 dict = 1.00 all = 0.2698 } ---------------------------- total value len = 0.790632 GB avg = 0.119 KB (by entry num) total key len = 0.185565 GB avg = 0.028 KB total ukey len = 0.132546 GB avg = 0.020 KB total ukey num = 0.006627312 Billion total entry num = 0.006627312 Billion write speed all = 0.023324226 MB/sec (with seq num) write speed all = 0.022057458 MB/sec (without seq num) 2024/11/22-20:29:56.488342 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226148: 923796 keys, 36216010 bytes, temperature: kWarm 2024/11/22-20:29:56.488409 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278596488374, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226148, "file_size": 36216010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3394785136, "largest_seqno": 3540923648, "table_properties": {"data_size": 30136464, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866288, "raw_average_key_size": 28, "raw_value_size": 108351536, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923796, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278592, "oldest_key_time": 0, "file_creation_time": 1732278592, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226148, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:56.526539 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst) /dev/shm/ToplingTemp/Topling-279411-wOuu4Y 2024/11/22-20:29:57.408733 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst): first pass time = 0.88's, 152.150'MB/sec 2024/11/22-20:29:58.737827 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst): old prealloc_size = 147639500, real_size = 32132832 2024/11/22-20:29:58.738369 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923851 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 117.28 avg-zval = 32.63 2024/11/22-20:29:58.739143 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:29:59.608354 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226149.sst): second pass time = 1.33's, 81.587'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.612'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3389.220'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 278.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32450.766'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.587'MB/sec zip pipeline throughput = 81.587'MB/sec entries = 923851 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 32.63 usrkeys = 923851 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.28 avg-zval = 32.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923851 bytes = 3233488 } seq expand size = 6466957 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5128 value = 3.5944 dict = 1.00 all = 3.7052 } Zip/UnZip{ index = 0.0951 value = 0.2782 dict = 1.00 all = 0.2699 } ---------------------------- total value len = 0.905449 GB avg = 0.120 KB (by entry num) total key len = 0.211433 GB avg = 0.028 KB total ukey len = 0.151023 GB avg = 0.020 KB total ukey num = 0.007551163 Billion total entry num = 0.007551163 Billion write speed all = 0.026683589 MB/sec (with seq num) write speed all = 0.025240341 MB/sec (without seq num) 2024/11/22-20:29:59.639483 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226149: 923851 keys, 36223790 bytes, temperature: kWarm 2024/11/22-20:29:59.639551 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278599639511, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226149, "file_size": 36223790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3540923338, "table_properties": {"data_size": 30144144, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25867828, "raw_average_key_size": 28, "raw_value_size": 108349998, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923851, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278596, "oldest_key_time": 0, "file_creation_time": 1732278596, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226149, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:29:59.678369 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst) /dev/shm/ToplingTemp/Topling-279411-jfSRjV 2024/11/22-20:30:00.595543 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst): first pass time = 0.92's, 146.345'MB/sec 2024/11/22-20:30:01.903596 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst): old prealloc_size = 147639500, real_size = 30975728 2024/11/22-20:30:01.904141 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922211 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0290 GB avg-val = 117.54 avg-zval = 31.42 2024/11/22-20:30:01.904899 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:30:02.751285 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226150.sst): second pass time = 1.31's, 82.911'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.361'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.921'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 272.078'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33032.471'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.911'MB/sec zip pipeline throughput = 82.911'MB/sec entries = 922211 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.54 avg-zval = 31.42 usrkeys = 922211 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.54 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922211 bytes = 3227760 } seq expand size = 6455477 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4455 value = 3.7405 dict = 1.00 all = 3.8282 } Zip/UnZip{ index = 0.0957 value = 0.2673 dict = 1.00 all = 0.2612 } ---------------------------- total value len = 1.020301 GB avg = 0.120 KB (by entry num) total key len = 0.237254 GB avg = 0.028 KB total ukey len = 0.169467 GB avg = 0.020 KB total ukey num = 0.008473374 Billion total entry num = 0.008473374 Billion write speed all = 0.030042177 MB/sec (with seq num) write speed all = 0.028422790 MB/sec (without seq num) 2024/11/22-20:30:02.783047 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226150: 922211 keys, 35060344 bytes, temperature: kWarm 2024/11/22-20:30:02.783111 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278602783078, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226150, "file_size": 35060344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3384247007, "largest_seqno": 3540923354, "table_properties": {"data_size": 28979232, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25821908, "raw_average_key_size": 28, "raw_value_size": 108395822, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922211, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278599, "oldest_key_time": 0, "file_creation_time": 1732278599, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226150, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:30:02.831266 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst) /dev/shm/ToplingTemp/Topling-279411-NGNeKW 2024/11/22-20:30:03.760906 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst): first pass time = 0.93's, 144.383'MB/sec 2024/11/22-20:30:05.092730 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst): old prealloc_size = 147639500, real_size = 33642848 2024/11/22-20:30:05.093309 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928679 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.53 avg-zval = 34.07 2024/11/22-20:30:05.094087 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:30:06.016152 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226151.sst): second pass time = 1.33's, 81.290'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.877'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.650'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 294.729'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32686.406'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.290'MB/sec zip pipeline throughput = 81.290'MB/sec entries = 928679 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 34.07 usrkeys = 928679 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 34.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928679 bytes = 3250384 } seq expand size = 6500753 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5134 value = 3.4198 dict = 1.00 all = 3.5558 } Zip/UnZip{ index = 0.0951 value = 0.2924 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 1.135016 GB avg = 0.121 KB (by entry num) total key len = 0.263257 GB avg = 0.028 KB total ukey len = 0.188041 GB avg = 0.020 KB total ukey num = 0.009402053 Billion total entry num = 0.009402053 Billion write speed all = 0.033401248 MB/sec (with seq num) write speed all = 0.031604517 MB/sec (without seq num) 2024/11/22-20:30:06.052057 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226151: 928679 keys, 37746528 bytes, temperature: kWarm 2024/11/22-20:30:06.052119 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278606052087, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226151, "file_size": 37746528, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3540923369, "table_properties": {"data_size": 31643856, "index_size": 1766656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003012, "raw_average_key_size": 28, "raw_value_size": 108214846, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928679, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278602, "oldest_key_time": 0, "file_creation_time": 1732278602, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226151, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:30:06.106547 140025221817920 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst) /dev/shm/ToplingTemp/Topling-279411-l7QsNW 2024/11/22-20:30:06.158023 140025221817920 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst): first pass time = 0.05's, 225.973'MB/sec 2024/11/22-20:30:06.262343 140025221817920 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst): old prealloc_size = 147639500, real_size = 3600736 2024/11/22-20:30:06.262458 140025221817920 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 80018 prefix = 4 raw-key = 0.0013 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0094 GB zip-val = 0.0034 GB avg-val = 117.25 avg-zval = 42.82 2024/11/22-20:30:06.262585 140025221817920 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.012 GB 2024/11/22-20:30:06.329179 140025221817920 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00/226152.sst): second pass time = 0.10's, 90.399'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.07's, 59.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1694.188'MB/sec (index lex order gen) rebuild zvType time = 0.06's, 0.362'MB/sec write SST data time = 0.01's, 346.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.097'MB, throughput = 3302.378'MB/sec zip my value time = 0.10's, unzip length = 0.009'GB zip my value throughput = 90.399'MB/sec zip pipeline throughput = 90.399'MB/sec entries = 80018 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.25 avg-zval = 42.82 usrkeys = 80018 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.25 avg-zval = 42.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 80018 bytes = 280080 } seq expand size = 560126 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.0094 GB dict = 0.10 MB all = 0.0116 GB } __ZipSize{ index = 0.0002 GB value = 0.0034 GB dict = 0.10 MB all = 0.0040 GB } UnZip/Zip{ index = 10.4017 value = 2.7379 dict = 1.00 all = 2.9352 } Zip/UnZip{ index = 0.0961 value = 0.3652 dict = 1.00 all = 0.3407 } ---------------------------- total value len = 1.144958 GB avg = 0.121 KB (by entry num) total key len = 0.265498 GB avg = 0.028 KB total ukey len = 0.189641 GB avg = 0.020 KB total ukey num = 0.009482071 Billion total entry num = 0.009482071 Billion write speed all = 0.033692008 MB/sec (with seq num) write speed all = 0.031879999 MB/sec (without seq num) 2024/11/22-20:30:06.337942 140025221817920 [compaction_job.cc:1948] [default] [JOB 10232] Generated table #226152: 80018 keys, 3959637 bytes, temperature: kWarm 2024/11/22-20:30:06.338000 140025221817920 EVENT_LOG_v1 {"time_micros": 1732278606337968, "cf_name": "default", "job": 10232, "event": "table_file_creation", "file_number": 226152, "file_size": 3959637, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3539410604, "table_properties": {"data_size": 3426704, "index_size": 153856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2240504, "raw_average_key_size": 28, "raw_value_size": 9381972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 80018, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732278606, "oldest_key_time": 0, "file_creation_time": 1732278606, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 226152, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:30:06.348578 140025221817920 [compaction_job.cc:703] job-10232: subcompact[0], size: 0.374138 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-10232/att-00 [226142,226143,226144,226145,226146,226147,226148,226149,226150,226151,226152] 2024/11/22-20:30:06.420344 140025221817920 (Original Log Time 2024/11/22-20:29:26.458687) EVENT_LOG_v1 {"time_micros": 1732278566458645, "job": 10232, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [226131], "files_L3": [226098, 226099, 226100, 226101, 226102, 226103, 226104, 226105, 226106, 226107], "score": 0.999782, "input_data_size": 388255382, "oldest_snapshot_seqno": 3540946695} 2024/11/22-20:30:06.420348 140025221817920 (Original Log Time 2024/11/22-20:30:06.419694) [compaction_job.cc:2022] [default] [JOB 10232] Compacted 1@2 + 10@3 files to L3 => 374138081 bytes 2024/11/22-20:30:06.420349 140025221817920 (Original Log Time 2024/11/22-20:30:06.420279) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 9.7 rd, 9.4 wr, level 3, files in(1, 10) out(11 +0 blob) MB in(16.2, 354.1 +0.0 blob) out(356.8 +0.0 blob), read-write-amplify(44.9) write-amplify(22.0) OK, records in: 9541300, records dropped: 59229 output_compression: Snappy 2024/11/22-20:30:06.420355 140025221817920 (Original Log Time 2024/11/22-20:30:06.420324) EVENT_LOG_v1 {"time_micros": 1732278606420304, "cf": "default", "job": 10232, "event": "compaction_finished", "compaction_time_micros": 39889899, "compaction_time_cpu_micros": 17030272, "output_level": 3, "num_output_files": 11, "total_output_size": 374138081, "num_input_records": 9541300, "num_output_records": 9482071, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 260838715, "file_range_sync_nanos": 0, "file_fsync_nanos": 9840, "file_prepare_write_nanos": 25612, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-20:30:06.548029 140025221817920 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732278566,"executesMs":40105,"compactionJobId":10232,"attempt":0,"compactionInputRawBytes":1362211225,"compactionInputZipBytes":388255382,"compactionOutputRawBytes":0,"compactionOutputZipBytes":374138081,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-aa167dcf5c65b47b0227ea23d01cd5a1-bf1892643ddc245e-00"}}, response = 2024/11/22-20:30:06.548104 140025221817920 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-10232/att-00: olev 3, work 40.106 s, result 6.369 ms, install 0.681 ms, input{raw 1.269 GiB zip 370.269 MiB}