Multi-Camera Calibration Using Far-Range Dual-LED Wand and Near-Range Chessboard Fused in Bundle Adjustment.

This paper presents a calibration approach for multiple synchronized global-shutter RGB cameras surrounding a large capture volume for 3D application. The calibration approach uses an active wand with two LED-embedded markers waved manually within the target capture volume. Data from the waving wand are combined with chessboard images taken at close range during each camera's intrinsic calibration, optimizing camera parameters via our proposed bundle adjustment method.

Anatomical-Marker-Driven 3D Markerless Human Motion Capture.

Marker-based motion capture (mocap) is a conventional method used in biomechanics research to precisely analyze human movement. However, the time-consuming marker placement process and extensive post-processing limit its wider adoption. Therefore, markerless mocap systems that use deep learning to estimate 2D keypoint from images have emerged as a promising alternative, but annotation errors in training datasets used by deep learning models can affect estimation accuracy.

Wireless Pressure Sensor Array Module for Sensorized Object.

To complement rehabilitation assessments that involve hand-object interaction with additional information on the grasping parameters, we sensorized an object with a pressure sensor array module that can generate a pressure distribution map. The module can be customized for cylindrical and cuboid objects with up to 1024 sensing elements and it supports the efficient transfer of data wirelessly at more than 30 Hz. Although the module uses inexpensive materials, it is sensitive to changes in pressure distribution.

Real-time Tracking of Handheld Object from Color or Depth Images.

Object tracking during rehabilitation could help a therapist to evaluate a patient's movement and progress. Hence, we present an image-based method for real-time tracking of handheld objects due to its ease of use and availability of color or depth cameras. We use an efficient projective point correspondence method and generalize the use of precomputed spare viewpoint information to allow real-time tracking of a rigid object.

Autonomous modeling of repetitive movement for rehabilitation exercise monitoring.

Background: Insightful feedback generation for daily home-based stroke rehabilitation is currently unavailable due to the inefficiency of exercise inspection done by therapists. We aim to produce a compact anomaly representation that allows a therapist to pay attention to only a few specific sections in a long exercise session record and boost their efficiency in feedback generation.

Methods: This study proposes a data-driven technique to model a repetitive exercise using unsupervised phase learning on an artificial neural network and statistical learning on principal component analysis (PCA).

Camera-based Hand Tracking using a Mirror-based Multi-view Setup.

Current clinical practice of measuring hand joint range of motion relies on a goniometer as it is inexpensive, portable, and easy to use, but it can only measure the static angle of a single joint at a time. To measure dynamic hand motion, a camera-based system that can perform markerless hand pose estimation is attractive, as the system is ubiquitous, low-cost, and non-contact. However, camera-based systems require line-of-sight, and tracking accuracy degrades when the joint is occluded from the camera view.

Automatic Inference of Rat's Hindlimb Trajectory to Synchronize with Forelimb Gait Through Phase.

Synchronous forelimb-hindlimb gait pattern is important to facilitate natural walking behavior of an injured rat with total transection. Since our ultimate research goal is to build a rehabilitation robotic system to simulate the natural walking pattern for spinalized rats, this research aims to address an immediate goal of automating the inference of the rat's hindlimb trajectory from its own forelimb movement. Our proposed method uses unsupervised learning to extract independent forelimb and hinblimb phases.

Hand and Object Segmentation from Depth Image using Fully Convolutional Network.

Semantic segmentation is an important step for hand and object tracking as subsequent tracking algorithms depend heavily on the accuracy of the segmented hand and object. However, current methods for hand and object segmentation are limited in the number of semantic labels, and lack of a large scale annotated dataset to train an end-to-end deep neural network for semantic segmentation. Thus, in this work, we present a framework for generating a publicly available synthetic dataset, that is targeted for upper limb rehabilitation involving hand-object interaction and uses it to train our proposed deep neural network.

Unsupervised Phase Learning and Extraction from Repetitive Movements.

Phase extraction from repetitive movements is one crucial part in various applications such as interactive robotics, physical rehabilitation, or gait analysis. However, pre-existing automatic phase extraction techniques are specific to a target movement due to some handcrafted-features. To make it more universal, a novel unsupervised-learning-based phase extraction technique is proposed.

Personalized Markerless Upper-Body Tracking with a Depth Camera and Wrist-Worn Inertial Measurement Units.

A markerless motion capture technique is proposed based on a fusion between a depth camera (Kinect V2) and a pair of wrist-worn inertial measurement units (IMU). The method creates a personalized articulated human mesh model from one depth image frame and uses that model to improve the accuracy of the upper-body joint tracking. The IMUs are useful as an additional clue for the arm tracking, especially during an occlusion.

Recovery of forearm occluded trajectory in Kinect using a wrist-mounted Inertial Measurement Unit.

Kinect sensor is a successful device that lets 3D human motion capture be used in a general residential setting. This work aims to fulfill some missing capabilities in Kinect, which are forearm orientation estimation and forearm tracking in occlusion. By using a wrist-mounted Inertial Measurement Unit and Kinect's built-in skeleton tracking, we have developed a fusion procedure that improves the upper limb motion tracking without adding too many obtrusive devices to the user.

An elderly fall detection using a wrist-worn accelerometer and barometer.

As the world population is growing toward an aging society, elderly fall becomes a serious problem. Automatic fall detection and alert systems could shorten their waiting time after a fall and mitigate its physical and mental negative consequences. This work proposes a method that integrates a 3-axis accelerometer and a barometer on a wrist-worn device for the fall detection task.