The booming haptic data significantly improve the users' immersion during multimedia interaction. As a result, the study of a Haptic-based Interaction System has attracted the attention of the multimedia community. To construct such a system, a challenging task is the synchronization of multiple sensorial signals that is critical to the user experience. Despite audio-visual synchronization efforts, there is still a lack of a haptic-aware multimedia synchronization model. In this work, we propose a timestamp-independent synchronization for haptic-visual signal transmission. First, we exploit the sequential correlations during delivery and playback of a haptic-visual communication system. Second, we develop a key sample extraction of haptic signals based on the force feedback characteristics and a key frame extraction of visual signals based on deep-object detection. Third, we combine the key samples and frames to synchronize the corresponding haptic-visual signals. Without timestamps in the signal flow, the proposed method is still effective and more robust in complicated network conditions. Subjective evaluation also shows a significant improvement of user experience with the proposed method.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC9331103 | PMC |
| http://dx.doi.org/10.3390/s22155502 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Haptic-Based Real-Time Platform for Microswarm Steering in a Multi-Bifurcation Vascular Network.
Nanomaterials (Basel)
November 2024
School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK.
The use of electromagnetic fields to control a collection of magnetic nanoparticles, known as a microswarm, has many promising applications. Current research often makes use of accurate but time-consuming simulations lacking real-time human input. On the contrary, human interaction is possible with a real-time simulator, allowing the collection of valuable user interaction data.
View Article and Find Full Text PDFTelepresence augmentation for visual and haptic guided immersive teleoperation of industrial manipulator.
ISA Trans
July 2024
State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou, 310027, China; Key Laboratory of Intelligent Robot for Operation and Maintenance of Zhejiang Province, Hangzhou, 311121, China; Ocean College, Zhejiang University, Zhoushan, 316021, China. Electronic address:
Teleoperation under human guidance has become an effective solution to extend human's reach in various environments. However, the teleoperation system still faces challenges of insufficient sense of both visual and haptic feedback from remote environments, which results in the inadequate guidance for the operator. In this paper, a visual/haptic integrated perception and reconstruction system (VHI-PRS) is developed to provide the operator with 3D visual information and effective haptic guidance.
View Article and Find Full Text PDFA robotic sensory system with high spatiotemporal resolution for texture recognition.
Nat Commun
November 2023
Department of Materials Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
Humans can gently slide a finger on the surface of an object and identify it by capturing both static pressure and high-frequency vibrations. Although modern robots integrated with flexible sensors can precisely detect pressure, shear force, and strain, they still perform insufficiently or require multi-sensors to respond to both static and high-frequency physical stimuli during the interaction. Here, we report a real-time artificial sensory system for high-accuracy texture recognition based on a single iontronic slip-sensor, and propose a criterion-spatiotemporal resolution, to corelate the sensing performance with recognition capability.
View Article and Find Full Text PDFA Timestamp-Independent Haptic-Visual Synchronization Method for Haptic-Based Interaction System.
Sensors (Basel)
July 2022
Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information, College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China.
The booming haptic data significantly improve the users' immersion during multimedia interaction. As a result, the study of a Haptic-based Interaction System has attracted the attention of the multimedia community. To construct such a system, a challenging task is the synchronization of multiple sensorial signals that is critical to the user experience.
View Article and Find Full Text PDFVirtual Reality-Based Intervention for Enhancing Upper Extremity Function in Children With Hemiplegic Cerebral Palsy: A Literature Review.
Cureus
January 2022
Community Physiotherapy, N.K.P. Salve Institute of Medical Sciences, Nagpur, IND.
Cerebral palsy (CP) is the most common cause of motor disability in the pediatric population, with hemiplegia as one of the most widely seen subtypes of spastic CP. Although most of the children with hemiplegic CP are independent ambulators, deficits in hand function of the affected side remain a major concern of caregivers and children themselves. Children use the unaffected upper extremity to compensate for the weakness in the affected one, which consequently leads to the disuse of the hemiparetic upper extremity.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!