Reinforcement Learning for Avionics Applications

Simulation-based optimization techniques that enlist reinforcement learning controllers are ideally suited for complex and multi-objective optimization problems that cannot be solved easily using traditional techniques, especially when the stochastic natures or the environment, resources, and external interactive entities are taken into account. Reinforcement learning based on incremental value iteration may be unstable when sequential system updates are close together and when general compact function approximators are required. Residual methods can preclude these problems. The proposed work will investigate and refine residual reinforcement learning techniques suitable for high-dimensional systems with many complex subsystem interactions and characterized by an aggregation of continuous, discrete, logical-element and binary states. The work shall demonstrate, via simulation, residual reinforcement learning solutions for providing optimal detection, classification, and prioritization of multiple targets through uninhabited air vehicle (UAV) intelligent sensor allocation strategies and flight path optimizations. In addition to immediate benefits for the U.S. Air Force and its UAV research, the proposed work will result in reinforcement learning methods that are directly applicable to numerous military and commercial-sector systems, including navigation and trajectory optimization, commercial air-traffic control, and general simulation-based optimization techniques for complex systems.