An Improved Transformer-Based Neural Machine Translation Strategy: Interacting-Head Attention.

Transformer-based models have gained significant advances in neural machine translation (NMT). The main component of the transformer is the multihead attention layer. In theory, more heads enhance the expressive power of the NMT model.