A Gradient-Aware Search Algorithm for Constrained Markov Decision Processes.

The canonical solution methodology for finite constrained Markov decision processes (CMDPs), where the objective is to maximize the expected infinite-horizon discounted rewards subject to the expected infinite-horizon discounted costs' constraints, is based on convex linear programming (LP). In this brief, we first prove that the optimization objective in the dual linear program of a finite CMDP is a piecewise linear convex (PWLC) function with respect to the Lagrange penalty multipliers. Next, we propose a novel, provably optimal, two-level gradient-aware search (GAS) algorithm which exploits the PWLC structure to find the optimal state-value function and Lagrange penalty multipliers of a finite CMDP.

Node Deployment of Marine Monitoring Networks: A Multiobjective Optimization Scheme.

The increasing demands for real-time marine monitoring call for the wide deployment of Marine Monitoring Networks (MMNs). The low-rate underwater communications over a long distance, long propagation delay of underwater acoustic channel, and high deployment costs of marine sensors in a large-scale three-dimensional space bring great challenges in the network deployment and management of MMN. In this paper, we first propose a multitier, hierarchical network architecture of MMN with the support of edge computing (HMMN-EC) to enable efficient monitoring services in a harsh marine environment, taking into consideration the salient features of marine communications.