Bridging vision and touch: advancing robotic interaction prediction with self-supervised multimodal learning.

Predicting the consequences of the agent's actions on its environment is a pivotal challenge in robotic learning, which plays a key role in developing higher cognitive skills for intelligent robots. While current methods have predominantly relied on vision and motion data to generate the predicted videos, more comprehensive sensory perception is required for complex physical interactions such as contact-rich manipulation or highly dynamic tasks. In this work, we investigate the interdependence between vision and tactile sensation in the scenario of dynamic robotic interaction.

Visuo-dynamic self-modelling of soft robotic systems.

Soft robots exhibit complex nonlinear dynamics with large degrees of freedom, making their modelling and control challenging. Typically, reduced-order models in time or space are used in addressing these challenges, but the resulting simplification limits soft robot control accuracy and restricts their range of motion. In this work, we introduce an end-to-end learning-based approach for fully dynamic modelling of any general robotic system that does not rely on predefined structures, learning dynamic models of the robot directly in the visual space.

Sensorized Skin With Biomimetic Tactility Features Based on Artificial Cross-Talk of Bimodal Resistive Sensory Inputs.

Tactility in biological organisms is a faculty that relies on a variety of specialized receptors. The bimodal sensorized skin, featured in this study, combines soft resistive composites that attribute the skin with mechano- and thermoreceptive capabilities. Mimicking the position of the different natural receptors in different depths of the skin layers, a multi-layer arrangement of the soft resistive composites is achieved.

Autonomous detection and sorting of litter using deep learning and soft robotic grippers.

Road infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task.

3D Printable Soft Sensory Fiber Networks for Robust and Complex Tactile Sensing.

The human tactile system is composed of multi-functional mechanoreceptors distributed in an optimized manner. Having the ability to design and optimize multi-modal soft sensory systems can further enhance the capabilities of current soft robotic systems. This work presents a complete framework for the fabrication of soft sensory fiber networks for contact localization, using pellet-based 3D printing of piezoresistive elastomers to manufacture flexible sensory networks with precise and repeatable performances.

Closing the Control Loop with Time-Variant Embedded Soft Sensors and Recurrent Neural Networks.

Embedded soft sensors can significantly impact the design and control of soft-bodied robots. Although there have been considerable advances in technology behind these novel sensing materials, their application in real-world tasks, especially in closed-loop control tasks, has been severely limited. This is mainly because of the challenge involved with modeling a nonlinear time-variant sensor embedded in a complex soft-bodied system.

Manipulation of free-floating objects using Faraday flows and deep reinforcement learning.

The ability to remotely control a free-floating object through surface flows on a fluid medium can facilitate numerous applications. Current studies on this problem have been limited to uni-directional motion control due to the challenging nature of the control problem. Analytical modelling of the object dynamics is difficult due to the high-dimensionality and mixing of the surface flows while the control problem is hard due to the nonlinear slow dynamics of the fluid medium, underactuation, and chaotic regions.

Soft Self-Healing Fluidic Tactile Sensors with Damage Detection and Localization Abilities.

Self-healing sensors have the potential to increase the lifespan of existing sensing technologies, especially in soft robotic and wearable applications. Furthermore, they could bestow additional functionality to the sensing system because of their self-healing ability. This paper presents the design for a self-healing sensor that can be used for damage detection and localization in a continuous manner.

Learning to stop: a unifying principle for legged locomotion in varying environments.

Evolutionary studies have unequivocally proven the transition of living organisms from water to land. Consequently, it can be deduced that locomotion strategies must have evolved from one environment to the other. However, the mechanism by which this transition happened and its implications on bio-mechanical studies and robotics research have not been explored in detail.

First-Order Dynamic Modeling and Control of Soft Robots.

Modeling of soft robots is typically performed at the static level or at a second-order fully dynamic level. Controllers developed upon these models have several advantages and disadvantages. Static controllers, based on the kinematic relations tend to be the easiest to develop, but by sacrificing accuracy, efficiency and the natural dynamics.

Emergence of behavior through morphology: a case study on an octopus inspired manipulator.

The complex motion abilities of the Octopus vulgaris have been an intriguing research topic for biologists and roboticists alike. Various studies have been conducted on the underlying control architectures employed by these high dimensional biological organisms. Researchers have attempted to replicate these architectures on robotic systems.

Soft robot perception using embedded soft sensors and recurrent neural networks.

Recent work has begun to explore the design of biologically inspired soft robots composed of soft, stretchable materials for applications including the handling of delicate materials and safe interaction with humans. However, the solid-state sensors traditionally used in robotics are unable to capture the high-dimensional deformations of soft systems. Embedded soft resistive sensors have the potential to address this challenge.

Modeling the Encoding of Saccade Kinematic Metrics in the Purkinje Cell Layer of the Cerebellar Vermis.

Recent electrophysiological observations related to saccadic eye movements in rhesus monkeys, suggest a prediction of the sensory consequences of movement in the Purkinje cell layer of the cerebellar oculomotor vermis (OMV). A definite encoding of real-time motion of the eye has been observed in simple-spike responses of the combined burst-pause Purkinje cell populations, organized based upon their complex-spike directional tuning. However, the underlying control mechanisms that could lead to such action encoding are still unclear.

Control Strategies for Soft Robotic Manipulators: A Survey.

With the rise of soft robotics technology and applications, there have been increasing interests in the development of controllers appropriate for their particular design. Being fundamentally different from traditional rigid robots, there is still not a unified framework for the design, analysis, and control of these high-dimensional robots. This review article attempts to provide an insight into various controllers developed for continuum/soft robots as a guideline for future applications in the soft robotics field.

Learning Closed Loop Kinematic Controllers for Continuum Manipulators in Unstructured Environments.

This article introduces a machine-learning-based approach for closed loop kinematic control of continuum manipulators in the task space. For this purpose, we propose a unique formulation for learning the inverse kinematics of a continuum manipulator while integrating end-effector feedback. We demonstrate that this model-free approach for kinematic control is very well suited for nonlinear stochastic continuum robots.

Learning dynamic models for open loop predictive control of soft robotic manipulators.

The soft capabilities of biological appendages like the arms of Octopus vulgaris and elephants' trunks have inspired roboticists to develop their robotic equivalents. Although there have been considerable efforts to replicate their morphology and behavior patterns, we are still lagging behind in replicating the dexterity and efficiency of these biological systems. This is mostly due to the lack of development and application of dynamic controllers on these robots which could exploit the morphological properties that a soft-bodied manipulator possesses.

Objective evaluation of oral presentation skills using Inertial Measurement Units.

Oral presentation is considered as one of the most sought after skills by companies and professional organizations and program accreditation agencies. However, both learning process and evaluation of this skill are time demanding and complex tasks that need dedication and experience. Furthermore, the role of the instructor is fundamental during the presentation assessment.