AI Article Synopsis
Article Abstract
With the emergence of Artificial Intelligence (AI), new attention has been given to implement AI algorithms on resource constrained tiny devices to expand the application domain of IoT. Multimodal Learning has recently become very popular with the classification task due to its impressive performance for both image and audio event classification. This paper presents - a flexible system algorithm co-designed multimodal learning framework for resource constrained tiny devices. The framework was designed to be evaluated on two different case-studies: COVID-19 detection from multimodal audio recordings and battle field object detection from multimodal images and audios. In order to compress the model to implement on tiny devices, substantial network architecture optimization and mixed precision quantization were performed (mixed 8-bit and 4-bit). shows that even a tiny multimodal learning model can improve the classification performance than that of any unimodal frameworks. The most compressed achieves 88.4% COVID-19 detection accuracy (14.5% improvement from unimodal base model) and 96.8% battle field object detection accuracy (3.9% improvement from unimodal base model). Finally, we test our models on a Raspberry Pi 4 to see how they perform when deployed to a resource constrained tiny device.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8845504 | PMC |
Publication Analysis
Top Keywords
Similar Publications
Multi-branch CNNFormer: a novel framework for predicting prostate cancer response to hormonal therapy.
Biomed Eng Online
December 2024
Department of Bioengineering, University of Louisville, Louisville, KY, USA.
Purpose: This study aims to accurately predict the effects of hormonal therapy on prostate cancer (PC) lesions by integrating multi-modality magnetic resonance imaging (MRI) and the clinical marker prostate-specific antigen (PSA). It addresses the limitations of Convolutional Neural Networks (CNNs) in capturing long-range spatial relations and the Vision Transformer (ViT)'s deficiency in localization information due to consecutive downsampling. The research question focuses on improving PC response prediction accuracy by combining both approaches.
View Article and Find Full Text PDFHarnessing multimodal approaches for depression detection using large language models and facial expressions.
Npj Ment Health Res
December 2024
Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, 91052, Germany.
Detecting depression is a critical component of mental health diagnosis, and accurate assessment is essential for effective treatment. This study introduces a novel, fully automated approach to predicting depression severity using the E-DAIC dataset. We employ Large Language Models (LLMs) to extract depression-related indicators from interview transcripts, utilizing the Patient Health Questionnaire-8 (PHQ-8) score to train the prediction model.
View Article and Find Full Text PDFMachine Learning and Deep Learning for Diagnosis of Lumbar Spinal Stenosis: Systematic Review and Meta-Analysis.
J Med Internet Res
December 2024
Beijing Chaoyang Hospital, Capital Medical University, Beijing, China.
Background: Lumbar spinal stenosis (LSS) is a major cause of pain and disability in older individuals worldwide. Although increasing studies of traditional machine learning (TML) and deep learning (DL) were conducted in the field of diagnosing LSS and gained prominent results, the performance of these models has not been analyzed systematically.
Objective: This systematic review and meta-analysis aimed to pool the results and evaluate the heterogeneity of the current studies in using TML or DL models to diagnose LSS, thereby providing more comprehensive information for further clinical application.
Coupling of state space modules and attention mechanisms: An input-aware multi-contrast MRI synthesis method.
Med Phys
December 2024
Jiangsu Key Laboratory for Biomaterials and Devices, School of Biological Science and Medical Engineering, Southeast University, Nanjing, China.
Background: Medical imaging plays a pivotal role in the real-time monitoring of patients during the diagnostic and therapeutic processes. However, in clinical scenarios, the acquisition of multi-modal imaging protocols is often impeded by a number of factors, including time and economic costs, the cooperation willingness of patients, imaging quality, and even safety concerns.
Purpose: We proposed a learning-based medical image synthesis method to simplify the acquisition of multi-contrast MRI.
Role of Community-Clinical Partnerships to Promote Cancer Screening: Lessons Learned From the National Breast and Cervical Cancer Early Detection Program.
Health Promot Pract
December 2024
Centers for Disease Control and Prevention, Chamblee, GA, USA.
Community-clinical partnerships are an effective approach to connecting primary care with public health to increase disease prevention and screenings and reduce health inequities. We explore how the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) award recipients and clinic teams are using community-clinical linkages to deliver services to populations who are without access to health care and identify barriers, facilitators, and lessons that can be used to improve program implementation. We used purposive sampling to select nine state recipients of the NBCCEDP and a clinic partner for each recipient.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!