Online Learning of Autonomous Helicopter Control

Abstract

This paper details the development of an online adaptive control system, designed to learn from the actions of an instructing pilot. Three learning architectures, single layer neural networks (SLNN), multi-layer neural networks (MLNN), and fuzzy associative memories (FAM) are considerd. Each method has been tested in simulation. While the SLNN and MLNN provided adequate control under some simulation conditions, the addition of pilot noise and pilot variation during simulation training caused these methods to fail. FAMs alone were found to be suitable under all simulation conditions. The performance of an FAM velocity control being used by a higher level waypoint controller in simulation is presented. As a first step toward performing real helicopter tests, an FAM attitude controller has also been developed in simulation, and its tracking performance presented. of the non-conventional studies have moved beyond simulations. While many of the neuro/fuzzy approaches incorporate some form of limited authority, online adaption, the majority of control authority is learnt offline. This offline controller learning has been performed using actor/critic type reinformcement learning [8], policy search methods based on MDP models built from flight data [2], and back propagation of errors through neural forward models [10]. Only two methods have used pilot control signals to directly train a feedback controller [14, 5]. While the FAMs in [14] showed promise in simulation, the controllers failed to stabilise the real helicopter. The MLNN in [5], on the other hand, managed to stabilise the real helicopter for brief periods, however later investigations showed that the system had many hidden, potential instabilities [3, 4]. 1