Focused learning by antibody language models using preferential masking of non-templated regions.

Existing antibody language models () are pre-trained using a masked language modeling () objective with uniform masking probabilities. While these models excel at predicting germline residues, they often struggle with mutated and non-templated residues, which are crucial for antigen-binding specificity and concentrate in the complementarity-determining regions (). Here, we demonstrate that preferential masking of the non-templated CDR3 is a compute-efficient strategy to enhance model performance.