A Model of Reaching that Integrates Continuous Reinforcement Learning, Accumulator Models, and Direct Inverse Modeling
Dimitri Ognibene, Angelo Rega and Gianluca Baldassarre
Simulation of Adaptive Behavior 2006 (SAB 2006)
Rome, Italy, 25-29 September 2006
Summary
The leading idea of this paper is that, in order to tackle novel complex behavioral tasks by trial-and-error learning, humans and monkeys rely upon repertoires of sensorimotor primitives that allow them to search solutions in a space coarser than the space of fine movements. How do they represent sensorimotor primitives in their brain, learn them, and assemble them in order to tackle those complex tasks? This paper gives a preliminary answer to these questions by presenting an architecture that integrates direct inverse modeling (that mimics how infants learn repertoires of sensorimotor primitives, based on postural goals), accumulator models (biologically plausible models of action se-lection) and actor-critic reinforcement learning (used here to mimic how adults assemble sensorimotor primitives to tackle reward-based complex tasks). The major novelties of the paper are as follows: a) integration of the aforementioned components in a whole architecture; b) improvement of the actor’s biological plausibility through an accumulator model; c) use of a reinforcement learning model to select sensorimotor primitives in a continuous space; d) training of the actor “to reach own hand” so to acquire a bias to reach salient points in space.