Human-Robot Collaboration With a Corrective Shared Controlled Robot in a Sanding Task.

Objective: Physical and cognitive workloads and performance were studied for a corrective shared control (CSC) human-robot collaborative (HRC) sanding task.

Background: Manual sanding is physically demanding. Collaborative robots (cobots) can potentially reduce physical stress, but fully autonomous implementation has been particularly challenging due to skill, task variability, and robot limitations.

OptiGait: Gait Monitoring Using An Ankle-Worn Stereo Camera System.

We developed an ankle-worn gait monitoring system for tracking gait parameters, including length, width, and height. The system utilizes ankle bracelets equipped with wide-angle infrared (IR) stereo cameras tasked with monitoring a marker on the opposing ankle. A computer vision algorithm we have also developed processes the imaged marker positions to estimate the length, width, and height of the person's gait.

Efficient online real-time video stabilization with a novel least squares formulation and parallel AC-RANSAC.

A novel online real-time video stabilization algorithm (LSstab) that suppresses unwanted motion jitters based on cinematography principles is presented. LSstab features a parallel realization of the RANSAC (AC-RANSAC) algorithm to estimate the inter-frame camera motion parameters. A novel least squares based smoothing cost function is then proposed to mitigate undesirable camera jitters according to cinematography principles.

AI-Based Automatic System for Assessing Upper-Limb Spasticity of Patients With Stroke Through Voluntary Movement.

Spasticity is a common complication for patients with stroke, but only few studies investigate the relation between spasticity and voluntary movement. This study proposed a novel automatic system for assessing the severity of spasticity (SS) of four upper-limb joints, including the elbow, wrist, thumb, and fingers, through voluntary movements. A wearable system which combined 19 inertial measurement units and a pressure ball was proposed to collect the kinematic and force information when the participants perform four tasks, namely cone stacking (CS), fast flexion and extension (FFE), slow ball squeezing (SBS), and fast ball squeezing (FBS).

Design of a Multi-Sensor System for Exploring the Relation between Finger Spasticity and Voluntary Movement in Patients with Stroke.

A novel wearable multi-sensor data glove system is developed to explore the relation between finger spasticity and voluntary movement in patients with stroke. Many stroke patients suffer from finger spasticity, which is detrimental to their manual dexterity. Diagnosing and assessing the degrees of spasticity require neurological testing performed by trained professionals to estimate finger spasticity scores via the modified Ashworth scale (MAS).

Comparison of the observer, single-frame video and computer vision hand activity levels.

Observer, manual single-frame video, and automated computer vision measures of the Hand Activity Level (HAL) were compared. HAL can be measured three ways: (1) observer rating (HAL), (2) calculated from single-frame multimedia video task analysis for measuring frequency (F) and duty cycle (D) (HAL), or (3) from automated computer vision (HAL). This study analysed videos collected from three prospective cohort studies to ascertain HAL, HAL, and HAL for 419 industrial videos.

Load Asymmetry Angle Estimation Using Multiple view Videos.

A robust computer vision-based approach is developed to estimate the load asymmetry angle defined in the revised NIOSH lifting equation (RNLE). The angle of asymmetry enables the computation of a recommended weight limit for repetitive lifting operations in a workplace to prevent lower back injuries. An open-source package OpenPose is applied to estimate the 2D locations of skeletal joints of the worker from two synchronous videos.

Towards real-time 3D visualization with multiview RGB camera array.

A real-time 3D visualization (RT3DV) system using a multiview RGB camera array is presented. RT3DV can process multiple synchronized video streams to produce a stereo video of a dynamic scene from a chosen view angle. Its design objective is to facilitate 3D visualization at the video frame rate with good viewing quality.

Estimating Trunk Angle Kinematics During Lifting Using a Computationally Efficient Computer Vision Method.

Objective: A computer vision method was developed for estimating the trunk flexion angle, angular speed, and angular acceleration by extracting simple features from the moving image during lifting.

Background: Trunk kinematics is an important risk factor for lower back pain, but is often difficult to measure by practitioners for lifting risk assessments.

Methods: Mannequins representing a wide range of hand locations for different lifting postures were systematically generated using the University of Michigan 3DSSPP software.

Large-Field-of-View Visualization with Small Blind Spots Utilizing Tilted Micro-Camera Array for Laparoscopic Surgery.

Existing laparoscopic surgery systems use a single laparoscope to visualize the surgical area with a limited field of view (FoV), necessitating maneuvering the laparoscope to search a target region. In some cases, the laparoscope needs to be moved from one surgical port to another one to detect target organs. These maneuvers would cause longer surgical time and degrade the efficiency of operation.

Multi-View Image Denoising Using Convolutional Neural Network.

In this paper, we propose a novel multi-view image denoising algorithm based on convolutional neural network (MVCNN). Multi-view images are arranged into 3D focus image stacks (3DFIS) according to different disparities. The MVCNN is trained to process each 3DFIS and generate a denoised image stack that contains the recovered image information for regions of particular disparities.

The accuracy of a 2D video-based lifting monitor.

A widely used risk prediction tool, the revised NIOSH lifting equation (RNLE), provides the recommended weight limit (RWL), but is limited by analyst subjectivity, experience, and resources. This paper describes a robust, non-intrusive, straightforward approach to automatically extract spatial and temporal factors necessary for the RNLE using a single video camera in the sagittal plane. The participant's silhouette is segmented by motion information and the novel use of a ghosting effect provides accurate detection of lifting instances, and hand and feet location prediction.

Using Surgeon Hand Motions to Predict Surgical Maneuvers.

Objective: This study explores how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations.

Background: Automatic computer vision recognition of surgical maneuvers (suturing, tying, and transition) could expedite video review and objective assessment of surgeries.

Method: We recorded hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations, and applied three machine learning techniques (decision trees, random forests, and hidden Markov models) to classify surgical maneuvers every 2 s (60 frames) of video.

Large-Field-of-View Visualization Utilizing Multiple Miniaturized Cameras for Laparoscopic Surgery.

The quality and the extent of intra-abdominal visualization are critical to a laparoscopic procedure. Currently, a single laparoscope is inserted into one of the laparoscopic ports to provide intra-abdominal visualization. The extent of this field of view (FoV) is rather restricted and may limit efficiency and the range of operations.

Predicting Sagittal Plane Lifting Postures From Image Bounding Box Dimensions.

Objective: A method for automatically classifying lifting postures from simple features in video recordings was developed and tested. We explored if an "elastic" rectangular bounding box, drawn tightly around the subject, can be used for classifying standing, stooping, and squatting at the lift origin and destination.

Background: Current marker-less video tracking methods depend on a priori skeletal human models, which are prone to error from poor illumination, obstructions, and difficulty placing cameras in the field.

Optimal Camera Pose and Placement Configuration for Maximum Field-of-View Video Stitching.

An optimal camera placement problem is investigated. The objective is to maximize the area of the field of view (FoV) of a stitched video obtained by stitching video streams from an array of cameras. The positions and poses of these cameras are restricted to a given set of selections.

Measuring exertion time, duty cycle and hand activity level for industrial tasks using computer vision.

Two computer vision algorithms were developed to automatically estimate exertion time, duty cycle (DC) and hand activity level (HAL) from videos of workers performing 50 industrial tasks. The average DC difference between manual frame-by-frame analysis and the computer vision DC was -5.8% for the Decision Tree (DT) algorithm, and 1.

Visualizing stressful aspects of repetitive motion tasks and opportunities for ergonomic improvements using computer vision.

Patterns of physical stress exposure are often difficult to measure, and the metrics of variation and techniques for identifying them is underdeveloped in the practice of occupational ergonomics. Computer vision has previously been used for evaluating repetitive motion tasks for hand activity level (HAL) utilizing conventional 2D videos. The approach was made practical by relaxing the need for high precision, and by adopting a semi-automatic approach for measuring spatiotemporal characteristics of the repetitive task.

Approach to simultaneously denoise and invert backscatter and extinction from photon-limited atmospheric lidar observations.

Atmospheric lidar observations provide a unique capability to directly observe the vertical column of cloud and aerosol scattering properties. Detector and solar-background noise, however, hinder the ability of lidar systems to provide reliable backscatter and extinction cross-section estimates. Standard methods for solving this inverse problem are most effective with high signal-to-noise ratio observations that are only available at low resolution in uniform scenes.

Wireless Sensor Array Network DoA Estimation from Compressed Array Data via Joint Sparse Representation.

A compressive sensing joint sparse representation direction of arrival estimation (CSJSR-DoA) approach is proposed for wireless sensor array networks (WSAN). By exploiting the joint spatial and spectral correlations of acoustic sensor array data, the CSJSR-DoA approach provides reliable DoA estimation using randomly-sampled acoustic sensor data. Since random sampling is performed at remote sensor arrays, less data need to be transmitted over lossy wireless channels to the fusion center (FC), and the expensive source coding operation at sensor nodes can be avoided.

Estimation of low-altitude moving target trajectory using single acoustic array.

An acoustic-signature based method of estimating the flight trajectory of low-altitude flying aircraft that only requires a stationary microphone array is proposed. This method leverages the Doppler shifts of engine sound to estimate the closest point of approach distance, time, and speed. It also leverages the acoustic phase shift over the microphone array to estimate the direction of arrival of the target.

Measuring elemental time and duty cycle using automated video processing.

A marker-less 2D video algorithm measured hand kinematics (location, velocity and acceleration) in a paced repetitive laboratory task for varying hand activity levels (HAL). The decision tree (DT) algorithm identified the trajectory of the hand using spatiotemporal relationships during the exertion and rest states. The feature vector training (FVT) method utilised the k-nearest neighbourhood classifier, trained using a set of samples or the first cycle.

Evaluation of Simulated Clinical Breast Exam Motion Patterns Using Marker-Less Video Tracking.

Objective: This study investigates using marker-less video tracking to evaluate hands-on clinical skills during simulated clinical breast examinations (CBEs).

Background: There are currently no standardized and widely accepted CBE screening techniques.

Methods: Experienced physicians attending a national conference conducted simulated CBEs presenting different pathologies with distinct tumorous lesions.

Evaluation of Hands-On Clinical Exam Performance Using Marker-less Video Tracking.

This study investigates the potential of using marker-less video tracking of the hands for evaluating hands-on clinical skills. Experienced family practitioners attending a national conference were recruited and asked to conduct a breast examination on a simulator that simulates different clinical presentations. Videos were made of the clinician's hands during the exam and video processing software for tracking hand motion to quantify hand motion kinematics was used.

The accuracy of conventional 2D video for quantifying upper limb kinematics in repetitive motion occupational tasks.

Marker-less 2D video tracking was studied as a practical means to measure upper limb kinematics for ergonomics evaluations. Hand activity level (HAL) can be estimated from speed and duty cycle. Accuracy was measured using a cross-correlation template-matching algorithm for tracking a region of interest on the upper extremities.