Model-Free λ-Policy Iteration for Discrete-Time Linear Quadratic Regulation.

This article presents a model-free λ -policy iteration ( λ -PI) for the discrete-time linear quadratic regulation (LQR) problem. To solve the algebraic Riccati equation arising from solving the LQR in an iterative manner, we define two novel matrix operators, named the weighted Bellman operator and the composite Bellman operator. Then, the λ -PI algorithm is first designed as a recursion with the weighted Bellman operator, and its equivalent formulation as a fixed-point iteration with the composite Bellman operator is shown.

Resilient and Robust Synchronization of Multiagent Systems Under Attacks on Sensors and Actuators.

Resilient and robust distributed control protocols for multiagent systems under attacks on sensors and actuators are designed. A distributed H control protocol is designed to attenuate the disturbance or attack effects. However, the H controller is too conservative in the presence of attacks.

Optimal Output Regulation of Linear Discrete-Time Systems With Unknown Dynamics Using Reinforcement Learning.

This paper presents a model-free optimal approach based on reinforcement learning for solving the output regulation problem for discrete-time systems under disturbances. This problem is first broken down into two optimization problems: 1) a constrained static optimization problem is established to find the solution to the output regulator equations (i.e.

Optimal and Autonomous Control Using Reinforcement Learning: A Survey.

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal and control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories.

Off-Policy Reinforcement Learning: Optimal Operational Control for Two-Time-Scale Industrial Processes.

Industrial flow lines are composed of unit processes operating on a fast time scale and performance measurements known as operational indices measured at a slower time scale. This paper presents a model-free optimal solution to a class of two time-scale industrial processes using off-policy reinforcement learning (RL). First, the lower-layer unit process control loop with a fast sampling period and the upper-layer operational index dynamics at a slow time scale are modeled.

Actor-Critic Off-Policy Learning for Optimal Control of Multiple-Model Discrete-Time Systems.

In this paper, motivated by human neurocognitive experiments, a model-free off-policy reinforcement learning algorithm is developed to solve the optimal tracking control of multiple-model linear discrete-time systems. First, an adaptive self-organizing map neural network is used to determine the system behavior from measured data and to assign a responsibility signal to each of system possible behaviors. A new model is added if a sudden change of system behavior is detected from the measured data and the behavior has not been previously detected.

Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data.

In this paper, an output-feedback solution to the infinite-horizon linear quadratic tracking (LQT) problem for unknown discrete-time systems is proposed. An augmented system composed of the system dynamics and the reference trajectory dynamics is constructed. The state of the augmented system is constructed from a limited number of measurements of the past input, output, and reference trajectory in the history of the augmented system.

Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems.

This paper presents a partially model-free adaptive optimal control solution to the deterministic nonlinear discrete-time (DT) tracking control problem in the presence of input constraints. The tracking error dynamics and reference trajectory dynamics are first combined to form an augmented system. Then, a new discounted performance function based on the augmented system is presented for the optimal nonlinear tracking problem.