Mutual knowledge distillation (MKD) is a technique used to transfer knowledge between multiple models in a collaborative manner. However, it is important to note that not all knowledge is accurate or reliable, particularly under challenging conditions such as label noise, which can lead to models that memorize undesired information. This problem can be addressed by improving the reliability of the knowledge source, as well as selectively selecting reliable knowledge for distillation. While making a model more reliable is a widely studied topic, selective MKD has received less attention. To address this, we propose a new framework called selective mutual knowledge distillation (SMKD). The key component of SMKD is a generic knowledge selection formulation, which allows for either static or progressive selection thresholds. Additionally, SMKD covers two special cases: using no knowledge and using all knowledge, resulting in a unified MKD framework. We present extensive experimental results to demonstrate the effectiveness of SMKD and justify its design.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7616547 | PMC |
| http://dx.doi.org/10.1109/IJCNN54540.2023.10191991 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
A variety of deep generative models have been adopted to perform functional protein generation. Compared to 3D protein design, sequence-based generation methods, which aim to generate amino acid sequences with desired functions, remain a major approach for functional protein generation due to the abundance and quality of protein sequence data, as well as the relatively low modeling complexity for training. Although these models are typically trained to match protein sequences from the training data, exact matching of every amino acid is not always essential.
View Article and Find Full Text PDFFirst report of causing winged leaf dieback disease on in Yunnan Province, China.
Plant Dis
January 2025
Southwest Forestry University College of Landscape and Horticulture, College of Landscape and Horticulture, Kunming, Yunnan, China;
Rhus chinensis, a deciduous tree of the genus Rhus (family Anacardiaceae), is widely cultivated in China for its medicinal, edible, and ornamental value (Zhang et al., 2022). In April 2022, symptoms of winged leaf dieback disease were observed at Southwest Forestry University in Kunming, Yunnan Province, China (E102°45'42.
View Article and Find Full Text PDFFedKD-CPI: Combining the federated knowledge distillation technique to accomplish synergistic compound-protein interaction prediction.
Methods
January 2025
School of Computer Science and Engineering, Central South University, Changsha 410083, China; Hunan Provincial Key Lab on Bioinformatics, Central South University, Changsha 410083, China.
Compound-protein interaction (CPI) prediction is critical in the early stages of drug discovery, narrowing the search space for CPIs and reducing the cost and time required for traditional high-throughput screening. However, CPI-related data are usually distributed across different institutions and their sharing is restricted because of data privacy and intellectual property rights. Constructing a scheme that enhances multi-institutional collaboration to improve prediction accuracy while protecting data privacy is essential.
View Article and Find Full Text PDFMultiview attention networks for fine-grained watershed categorization via knowledge distillation.
PLoS One
January 2025
College of Landscape Architecture and Art, Jiangxi Agricultural University, Nanchang, China.
With the rapid development of artificial intelligence technology, an increasing number of village-related modeling problems have been addressed. However, first, the exploration of village-related watershed fine-grained classification problems, particularly the multi-view watershed fine-grained classification problem, has been hindered by dataset collection limitations; Second, village-related modeling networks typically employ convolutional modules for attentional modeling to extract salient features, yet they lack global attentional feature modeling capabilities; Lastly, the extensive number of parameters and significant computational demands render village-related watershed fine-grained classification networks infeasible for end-device deployment. To tackle these challenges, we introduce a multi-view attention mechanism designed for precise watershed classification, leveraging knowledge distillation techniques, abbreviated as MANet-KD.
View Article and Find Full Text PDFFirst Report of Causing Root Rot on Tulip Poplar in Tennessee and the United States.
Plant Dis
January 2025
Tennessee State University, Otis Floyd Nursery Research Center, 472 Cadillac Lane, McMinnville, Tennessee, United States, 37110;
Tulip poplar () is a member of the Magnolia family, is a large, fast-growing, long-lived, deciduous tree native to eastern North America. One-year-old tulip poplar seedlings grown under field conditions in a commercial nursery in Warren County, Tennessee, exhibited severe root rot in May 2024. Dark brown to black lesions were observed on the affected roots.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!