From Binary to Multi-Class Classification: A Two-Step Hybrid CNN-ViT Model for Chest Disease Classification Based on X-Ray Images.

Background/objectives: Chest disease identification for Tuberculosis and Pneumonia diseases presents diagnostic challenges due to overlapping radiographic features and the limited availability of expert radiologists, especially in developing countries. The present study aims to address these challenges by developing a Computer-Aided Diagnosis (CAD) system to provide consistent and objective analyses of chest X-ray images, thereby reducing potential human error. By leveraging the complementary strengths of convolutional neural networks (CNNs) and vision transformers (ViTs), we propose a hybrid model for the accurate detection of Tuberculosis and for distinguishing between Tuberculosis and Pneumonia.

Methods: We designed a two-step hybrid model that integrates the ResNet-50 CNN with the ViT-b16 architecture. It uses the transfer learning on datasets from for Pneumonia cases and datasets from for Tuberculosis cases. CNNs capture hierarchical structures in images, while ViTs, with their self-attention mechanisms, excel at identifying relationships between features. Combining these approaches enhances the model's performance on binary and multi-class classification tasks.

Results: Our hybrid CNN-ViT model achieved a binary classification accuracy of 98.97% for Tuberculosis detection. For multi-class classification, distinguishing between Tuberculosis, viral Pneumonia, and bacterial Pneumonia, the model achieved an accuracy of 96.18%. These results underscore the model's potential in improving diagnostic accuracy and reliability for chest disease classification based on X-ray images.

Conclusions: The proposed hybrid CNN-ViT model demonstrates substantial potential in advancing the accuracy and robustness of CAD systems for chest disease diagnosis. By integrating CNN and ViT architectures, our approach enhances the diagnostic precision, which may help to alleviate the burden on healthcare systems in resource-limited settings and improve patient outcomes in chest disease diagnosis.