The importance of experience replay database composition in deep reinforcement learning

Conference paper (2015)

Authors

T.D. de Bruin OLD Intelligent Control & Robotics
J. Kober OLD Intelligent Control & Robotics
K.P. Tuyls OLD Intelligent Control & Robotics , University of Liverpool
R. Babuska OLD Intelligent Control & Robotics
Research Group
OLD Intelligent Control & Robotics
More Info
expand_more

Published Date

2015

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Research Group

OLD Intelligent Control & Robotics

Abstract

Recent years have seen a growing interest in the use of deep neural networks as
function approximators in reinforcement learning. This paper investigates the potential of the Deep Deterministic Policy Gradient method for a robot control problem both in simulation and in a real setup. The importance of the size and composition of the experience replay database is investigated and some requirements on the distribution over the state-action space of the experiences in the database are identified. Of particular interest is the importance of negative experiences that are not close to an optimal policy. It is shown how training with samples that are insufficiently spread over the state-action space can cause the method to fail, and how maintaining the distribution over the state-action space of the samples in the experience database can greatly benefit learning.