Learning to Control a
Low-Cost Manipulator using Data-Efficient Reinforcement
Learning
International Conference on Robotics: Science & Systems (R:SS), 2011
Abstract
Over the last years, there has been substantial
progress in robust manipulation in unstructured environments.
The long-term goal of our work is to get away from precise,
but very expensive robotic systems and to develop affordable,
potentially imprecise, self-adaptive manipulator systems that can
interactively perform tasks such as playing with children. In
this paper, we demonstrate how a low-cost off-the-shelf robotic
system can learn closed-loop policies for a stacking task in only
a handful of trials—from scratch. Our manipulator is inaccurate
and provides no pose feedback. For learning a controller in the
work space of a Kinect-style depth camera, we use a model-based
reinforcement learning technique. Our learning method is data
efficient, reduces model bias, and deals with several noise sources
in a principled way during long-term planning. We present a
way of incorporating state-space constraints into the learning
process and analyze the learning gain by exploiting the sequential
structure of the stacking task.