Autors. Daniele Caligiore, Domenico Parisi, Gianluca Baldassarre
Topic and its relevance. Reaching, i.e. the capacity to get own hands in contact with objects in the environment, is a fundamental motor skill for primates and humans, at the basis of their capacity to interact with, manipulate, and change the world at own benefit. Here we use computational models to understand the mechanisms underlying learning and development of such reaching skill. The study of reaching is supported by the availability of a wealth of empirical data against which models can be tested.
Questions and goals. How can direct-inverse learning and trial-and-error learning lead to develop the reaching skill? What are the stages of development of reaching and how are they characterised in terms of kinematics and dynamics? How can reaching support more complex behaviours such sequences of button presses and obstacle avoidance?
Methods. We develop various computational models in the years. Some models explore in an abstract way the mechanisms for learning reaching, while others are closely compared with empirical data. Some models are based on self-supervised learning while others on reinforcement learning. Some models study the acquisition of simple-point-to point reaching movemenst while other study movements going around obstacles or sequences of reaching movements. All models are based on firing-rate neural networks and are tested within kinematic or dynamic simulated bodies (robots) of various complexity and realism.
Results. We show how relatively simple reaching skills can be acquired with direct-inverse learning while more sophisticated ones need trial-and-error learning. We show the conditions in which trial-and-error learning can support the emergence of rather sophisticated reaching movements. We account for some empirical data on more sophisticated reaching movements in humans and monkeys. We account for a wealth of kinematic/dynamic data that characterised different developmental stages of reaching as revealed in children longitudinal studies, at the same time isolating few key mechanisms behind such processes.
Conclusions. While simple associative learning can support some initial learning skills, trial-and-error learning is needed to acquire the most sophisticated reaching skills. The full account of the wealth of data on reaching development requires the addition to trial-and-error of few other key elements: the equilibrium point-based properties of muscles; the hypothesis on the need for accuracy of the end movement; the presence of muscular noise.
- Caligiore, D., Parisi, D., & Baldassarre, G. (2014). Integrating reinforcement learning, equilibrium points, and minimum variance to understand the development of reaching: A computational model. Psychological Review, 121(3), 389.