Hierarchical reinforcement learning and central pattern generators for modeling the development of rhythmic manipulation skills

The development of manipulation skills is a fundamental process for young primates as it leads them to acquire the capacity to modify the world to their advantage. As other motor skills, manipulation develops on the basis of motor babbling processes which are initially heavily based on the production of rhythmic movements. We propose a computational bio-inspired model to investigate the development of functional rhythmic hand skills from initially unstructured movements. The model is based on a hierarchical reinforcement-learning actor-critic model that searches the parameters of a set of central pattern generators (CPGs) having different degrees of sophistication. The model is tested with a simulated robotic hand engaged in rotating bottle cap-like objects having different shape and size. The results show that the model is capable of developing skills based on different combinations of CPGs so as to suitably manipulate the different objects. Overall, the model shows to be a valuable tool for the study of the development of rhythmic manipulation skills in primates.

Publication type: 
Contributo in atti di convegno
Author or Creator: 
Ciancio, Anna Lisa
Zollo, Loredana
Guglielmelli, Eugenio
Caligiore, Daniele
Baldassarre, Gianluca
IEEE Computer Society, Los Alamitos [CA], USA
First Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, pp. E1–8, Frankfurt am Main, Germany, 24-27 August 2011
Resource Identifier: 
ISTC Author: