Parameterized MDPs and Reinforcement Learning Problems-A Maximum Entropy Principle-Based Framework.

We present a framework to address a class of sequential decision-making problems. Our framework features learning the optimal control policy with robustness to noisy data, determining the unknown state and action parameters, and performing sensitivity analysis with respect to problem parameters. We consider two broad categories of sequential decision-making problems modeled as infinite horizon Markov decision processes (MDPs) with (and without) an absorbing state.

Robust atomic force microscopy using multiple sensors.

Atomic force microscopy typically relies on high-resolution high-bandwidth cantilever deflection measurements based control for imaging and estimating sample topography and properties. More precisely, in amplitude-modulation atomic force microscopy (AM-AFM), the control effort that regulates deflection amplitude is used as an estimate of sample topography; similarly, contact-mode AFM uses regulation of deflection signal to generate sample topography. In this article, a control design scheme based on an additional feedback mechanism that uses vertical z-piezo motion sensor, which augments the deflection based control scheme, is proposed and evaluated.

Fast and robust control of nanopositioning systems: Performance limits enabled by field programmable analog arrays.

This paper aims at control design and its implementation for robust high-bandwidth precision (nanoscale) positioning systems. Even though modern model-based control theoretic designs for robust broadband high-resolution positioning have enabled orders of magnitude improvement in performance over existing model independent designs, their scope is severely limited by the inefficacies of digital implementation of the control designs. High-order control laws that result from model-based designs typically have to be approximated with reduced-order systems to facilitate digital implementation.

Model based control of dynamic atomic force microscope.

A model-based robust control approach is proposed that significantly improves imaging bandwidth for the dynamic mode atomic force microscopy. A model for cantilever oscillation amplitude and phase dynamics is derived and used for the control design. In particular, the control design is based on a linearized model and robust H(∞) control theory.

Measuring robustness of the postural control system to a mild impulsive perturbation.

We propose a new metric to assess robustness of the human postural control system to an impulsive perturbation (in this case, a mild backward impulse force at the pelvis). By applying concepts from robust control theory, we use the inverse of the maximum value of the system's sensitivity function (1/MaxSens) as a measure for robustness of the human postural control system, e.g.

Robust broadband nanopositioning: fundamental trade-offs, analysis, and design in a two-degree-of-freedom control framework.

This paper studies and analyses fundamental trade-offs between positioning resolution, tracking bandwidth, and robustness to modeling uncertainties in two-degree-of-freedom (2DOF) control designs for nanopositioning systems. The analysis of these systems is done in optimal control setting with various architectural constraints imposed on the 2DOF framework. In terms of these trade-offs, our analysis shows that the primary role of feedback is providing robustness to the closed-loop device whereas the feedforward component is mainly effective in overcoming fundamental algebraic constraints that limit the feedback-only designs.

Feedback based simultaneous correction of imaging artifacts due to geometrical and mechanical cross-talk and tip-sample stick in atomic force microscopy.

This paper presents a feedback scheme that simultaneously corrects, in real time, for the imaging artifacts caused by cantilever and photosensor misalignments as well as misinterpretations in relative lateral position of the tip with respect to the sample due to the tip-sample stick in atomic force microscopy (AFM). The optical beam bounce method, typically used in AFM for imaging, is sensitive to inaccuracies of cantilever geometry and the relative misalignment of the laser source, cantilever, and the laser sensitive diode from the intended design. These inaccuracies, which contribute to the geometrical cross-talk between the normal and the lateral signals, become prominent at the atomic and subnanometer scales, and thereby impede high resolution imaging studies.