Skip to main content Skip to main navigation

Publikation

Evolving Neural Networks for Online Reinforcement Learning

Jan Hendrik Metzen; Mark Edgington; Yohannes Kassahun; Frank Kirchner
In: G. Rudolph; T. Jansen; S.M. Lucas; et al. (Hrsg.). Proceedings of the 10th International Conference on Parallel Problem Solving From Nature. International Conference on Parallel Problem Solving From Nature (PPSN-2008), September 13-17, Dortmund, Germany, Pages 518-527, Lecture Notes in Computer Science (LNCS), Vol. 5199, Springer, 2008.

Zusammenfassung

For many complex Reinforcement Learning problems with large and continuous state spaces, neuroevolution (the evolution of artificial neural networks) has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and evaluation phase of the system are separated. In contrast, in online Reinforcement Learning tasks where the actual performance of the systems during its learning phase matters, the results of neuroevolution are significantly impaired by its purely exploratory nature, meaning that it does not use (i. e. exploit) its knowledge of the performance of single individuals in order to improve its performance during learning. In this paper we describe modifications which significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies (EANT) and discuss the results obtained on two benchmark problems.