XElemNet: towards explainable AI for deep neural networks in materials science.

Recent progress in deep learning has significantly impacted materials science, leading to accelerated material discovery and innovation. ElemNet, a deep neural network model that predicts formation energy from elemental compositions, exemplifies the application of deep learning techniques in this field. However, the "black-box" nature of deep learning models often raises concerns about their interpretability and reliability.