The state-dependent Riccati equation (SDRE) technique can be used to solve optimal control problems for a wide class of nonlinear dynamical systems. In this method, instead of solving a complicated Hamilton-Jacobi-Bellman (HJB) equation, a state-dependent Riccati equation is solved which leads to a suboptimal control law. However, a priori model of the system must be available to apply this technique to the optimal control problem. In this paper, to solve the state-dependent Riccati equation without using a priori model of the system, a direct adaptive suboptimal algorithm is proposed. The algorithm, named state-dependent Riccati equation adaptive dynamic programing (SDRE-ADP), is based on a reinforcement learning approach which can be implemented in an online fashion. Like the SDRE technique, the proposed SDRE-ADP can stabilize the closed loop system locally asymptotically provided that some conditions are satisfied. Application of the proposed algorithm to an autonomous underwater vehicle (AUV) and also numerical simulation results show that it can be effectively applied for nonlinear systems.
