In this paper, we dive into sign language recognition, focusing on the recognition of isolated signs. The task is defined as a classification problem, where a sequence of frames (i.e., images) is recognized as one of the given sign language glosses. We analyze two appearance-based approaches, I3D and TimeSformer, and one pose-based approach, SPOTER. The appearance-based approaches are trained on a few different data modalities, whereas the performance of SPOTER is evaluated on different types of preprocessing. All the methods are tested on two publicly available datasets: AUTSL and WLASL300. We experiment with ensemble techniques to achieve new state-of-the-art results of 73.84% accuracy on the WLASL300 dataset by using the CMA-ES optimization method to find the best ensemble weight parameters. Furthermore, we present an ensembling technique based on the Transformer model, which we call Neural Ensembler.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC9269724 | PMC |
| http://dx.doi.org/10.3390/s22135043 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Effect of sign language learning on temporal resolution of visual attention.
J Vis
January 2025
Department of Communicative Disorders, University of Alabama, Tuscaloosa, AL, USA.
The visual environment of sign language users is markedly distinct in its spatiotemporal parameters compared to that of non-signers. Although the importance of temporal and spectral resolution in the auditory modality for language development is well established, the spectrotemporal parameters of visual attention necessary for sign language comprehension remain less understood. This study investigates visual temporal resolution in learners of American Sign Language (ASL) at various stages of acquisition to determine how experience with sign language affects perceptual sampling.
View Article and Find Full Text PDFSign recognition: the effect of parameters and features in sign mispronunciations.
Linguist Vanguard
December 2024
Laboratoire de Sciences Cognitives et Psycholinguistique (ENS, EHESS, CNRS), Ecole Normale Supérieure - PSL, 29 rue d'Ulm, 75005 Paris, France.
We investigate the degree to which mispronounced signs can be accommodated by signers of French Sign Language (LSF). Using an offline judgment task, we examine both the individual contributions of three parameters - handshape, movement, and location - to sign recognition, and the impact of the individual features that were manipulated to obtain the mispronounced signs. Results indicate that signers judge mispronounced handshapes to be less damaging for well-formedness than mispronounced locations or movements.
View Article and Find Full Text PDFOptimizing enzyme thermostability by combining multiple mutations using protein language model.
mLife
December 2024
State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic & Developmental Sciences, School of Life Sciences and Biotechnology Shanghai Jiao Tong University Shanghai China.
Article Synopsis
- Optimizing enzyme thermostability is crucial for protein science and industry, but combining multiple mutations can lead to inactivation, making traditional methods slow and inefficient.
- Researchers developed an AI-driven method to enhance enzyme thermostability by efficiently recombining beneficial single-point mutations, using data from various mutant groups.
- After two design rounds, the study achieved 50 combinatorial mutants with 100% success, including one exceptional mutant that significantly increased melting temperature and half-life, while also revealing complex interactions (epistasis) among mutations.
Diabetes Care Disparities in Deaf/Hard of Hearing and Blind/Low Vision Populations.
Curr Diab Rep
December 2024
College of Nursing, University of Utah, 10 South 2000 East, Salt Lake City, UT, 84112, USA.
Purpose Of Review: Describe the connection between Deaf/hard of hearing (DHH) and diabetes, explain the bidirectional relationship of blind/low vision (BLV) and diabetes, characterize challenges DHH and BLV populations face when seeking healthcare regarding their diabetes management. Highlight the inaccessibility of diabetes technology in these populations. Provide best practices when communicating with DHH and BLV people in the clinical setting.
View Article and Find Full Text PDFIntelligent Gesture Recognition Gloves for Real-Time Monitoring in Wireless Human-Computer Interaction.
ACS Appl Mater Interfaces
December 2024
National Engineering Lab of Special Display Technology, Special Display and Imaging Technology Innovation Center of Anhui Province, Academy of Optoelectronic Technology, Hefei University of Technology, Hefei 230009, China.
Flexible sensors mimic the sensing ability of human skin, and have unique flexibility and adaptability, allowing users to interact with intelligent systems in a more natural and intimate way. To overcome the issues of low sensitivity and limited operating range of flexible strain sensors, this study presents a highly innovative preparation method to develop a conductive elastomeric sensor with a cracked thin film by combining polydimethylsiloxane (PDMS) with multiwalled carbon nanotubes (MCNT). This novel design significantly increases both the sensitivity and operating range of the sensor (strain range 0-50%; the maximum tensile sensitivity of this sensor reaches 4.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!