Laboratory of Computational Embodied Neuroscience

OVERVIEW AND Focus Web-Pages on LOCEN Research Threads

Table: below we report a table listing the main research threads of LOCEN, organised by two main themes:
(1) IMOD: Intrinsic Motivations and Open-ended Development in robots and animals
(2) GAH: Goals, Actions, and Hierarchies in robots and animals
The research threads are also organised by their main focus (but most threads are highly interdisciplinary and involve more approaches):
(a) Robots;
(b) Behaviour;
(c) Brain.

Overview: below the table we overview each research thread, and we give the link to a focus web-page containing more explanations and material (we are working on them to add detailed explanations, pictures, videos, list relevant publications, etc. ). 

Papers: The references and pdf files of publications related to the research threads can be retrieved from this web-site here: 
Publications of LOCEN, including pdf files, by year 

Laboratories: The work of the group is based on three labs:

• iCubLab: Laboratory of Robotics
• MechLab: Laboratory of Mechatronics
• SimuLab: Laboratory of Computer Simulations

ROBOTS: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

IM-CLeVeR EU Project: Intrinsically Motivated Cumulative Learning Versatile Robots

Focus page: Further details on this research thread
Project dedicated page: IM-CLeVeR 
Authors: Vieri Santucci, Daniele Caligiore, Kristsana Seepanomwan, Marco Mirolli, Gianluca Baldassarre (but the project involved most LOCEN)
Topic and its relevance. This research thread is relative to the robotic works carried out within the IM-CLeVeR EU funded project. Developing robots able to autonomously discover, select, and solve multiple new tasks in a cumulative open-endeed fashion is an important issue for autonomous robotics. It becomes even crucial if we want to build robots capable of solving multiple problems in real environments posing challenges that are unknown at design time. There are two key `ingredients' necessary to build these kind of robots. The first are intrinsic motivations (IMs): these can drive autonomous learning of robots in an open-ended fashion in the absence of tasks assigned to the robots by the users. The second are hierarchical architectures: these are needed to store multiple skills, drive their acquistion with IMs, learn goals related to skills, and form complex skills based on simpler skills.

Active vision and open-ended learning

Focus page: Further details on this research thread 
Authors: Dimitri Ognibene, Valerio Sperati, Rodolfo Marraffa, Gianluca Baldassarre
Topic and its relevance. This project (initially funded by the EU project MindRACES and later by the EU project IM-CLeVeR) is on LOCEN's approach to vision called `Ecological Active vision - EAV'. EAV is grounded on the `active vision' approach, based on an actively-moved small fovea plus a low-resolution periphery, augmented with four principles: (a) a strong coupling of bottom-up and top-down attention processes; (b) the use of reinforcement-learning to acquire top-down attention skills; (c) the use of attention and vision to support pragmatic action (e.g., reaching and grasping) rather than vision per-se, in particular a strong spatial coupling between attention and manipulation actions; (d) the use of a novel Potential Action Memory component to collect information on the best places to visit with the fovea. Lately we have linked EAV with intrinsic motivations (IMs), in particular IM related to the perception of movement in the world and agency (i.e., the agent's perception of the capacity to cause movement in the world with own actions).

ROBOTS: goals, Actions, Hierarchies

Robots learning multiple skills (rhythmic/discrete motor actions, hierarchy, RL, transfer learning)

Focus page: Further details on this research thread
Authors: Daniele Caligiore, Paolo Tommasino, Annalisa Ciancio, Valentina Meola, Gianluca Baldassarre
Topic and its relevance: Building architectures that allow robots to learn multiple sensorimotor skills, possibly transferring knowledge between them, is a central open challenge for autonomous robotics. In particular, it is paramount to produce autonomous cumulative learning robots. This is also important to suggest possible architectures and processes through which brain solves the same problems.

Echo-state networks and dynamic motor behaviour: robots and brain

Focus page: Further details on this research thread
Authors: Francesco Mannella, Gianluca Baldassarre
Topic and its relevance. This research thread, started recently, concerns the use of echo-state networks and the modulation of their dynamics. Echo-state networks are an important class of neural networks belonging to the family of models called ``dynamic reservoires''. Echo-state networks have very interesting and powerful computational properties that make them suitable to learn and produce complex motor behaviours relevant for both robotics and for the study of motor behavoiur produced by brain. On the robotic side, the importance of this resides in the fact that the learning and production of sophisticated discrete and rhythmic movements is a pivotal building block of autonomous robotics architectures. In this respect, our approach is usable to face problems for which the autonomous robotics litearture uses devices such as Dynamic Movement Primitives (DMPs). The advantages of our approach with respect to DMPs is expected to be in terms of sophistication and flexibility of the movements producible with echo-state networks. The work is also important for brain modelling, in particular to model cortex viewed as a dynamical system whose dynamics is regulated by basal ganglia. The importance of this resides in the fact that the basal-ganglia and cortex form segregated loops that are a fundamental building module underlying multiple brain processes, from associative sensory processing, to motor behaviour, thinking, plannig, and reasoning. 

Collective navigation robotics

Focus page: Further details on this research thread
External dedicated web-page: Collective navigating robots
Authors: Gianluca Baldassarre, Domenico Parisi, Stefano Nolfi
Topic and its relevance. This is research thread, now terminated, was conducted within the EU funded project Swarm-bots. Collective robotics involves the use of multiple robots to carry out tasks that could not be carried out by single robots alone. For some tasks, the simplicity of single robots in terms of sensors, actuators, and communication capabilities can give robustness and low-cost to the whole ``swarm'' of robots. In this cases, the coordination between robots, needed to carry out a common task in cooperation, can rely on distributed (vs. centralised/hierarchical) coordination and communication mechanisms typically exploited by social insects (e.g., ants and bees), for example stigmergy (what one robot does with its body and in the environment is directly exploited by the others for coordination).

BEHAVIOUR: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

Intrinsic motivations: theory and empirical experiments

Focus page: Further details on this research thread
Authors: Vieri Santucci, Daniele Caligiore, Magda Mustile, Marco Mirolli, Gianluca Baldassarre
Topic and its relevance. Intrinsic motivations (IMs) are related to curisity, exploration, the interest for novel objects and surprising evens, and the drive to learn motor skill. IMs operate in the absence of a direct biological pressure and feedback (as in the case of extrinsic motivations, i.e. the classic motivations related to homeostatic regulations and survival). IMs are a fundametnal topic of investigation as they play a key role in human well being, art, science, and technology. They are also important for autonomous robotics as they allow the construction of cumulative learning robots. We are elaborating a general theory on intrinsic motivations. This offer predictions, testable in empirical experiments (examples are reported in this research thread), and mechanisms for computational models (see other research threads).

+Me: a wearable interactive device for Austism Spectrum Disorders 

Focus page: Further details on this research thread

External dedicated web-page: +me: Motivating children with Autism Spectrum Disorders to interact socially through the use of Transitional Wearable Companions. 

 Authors: Beste Ozcan, Valerio Sperati, Tania Moretta, Laura Romano, Daniele Caligiore, Gianluca Baldassarre (ISTC), Simone Scaffaro, Alessandro Medda (INI Villa Dante)
Topic and its relevance. Autism Spectrum Disorders -ASD- are a set of neurodevelopmental conditions characterised by the impairment, in varying degrees, of three basic areas for the psychic development of children: the social interaction; the communication(both verbal and not); the repertoire of activities and interests. This project is direced to realise an interactive wearable device that can potentially support and motivate the development of basic social skills.

BEHAVIOUR: GOALS, ACTIONS, HIERARCHIES

Children's development of reaching skills: an embodied model (iCub simulator)

Focus page: Further details on this research thread
Authors: Daniele Caligiore, Domenico Parisi, Gianluca Baldassarre
Topic and its relevance: Reaching, i.e. the capacity to get own hands in contact with objects in the environment, is a fundamental motor skill for primates and humans, at the basis of their capacity to interact with, manipulate, and change the world at own benefit. Here we use computational models to understand the mechanisms underlying learning and development of such reaching skill. The study of reaching is supported by the availability of a wealth of empirical data against which models can be tested. The model is mainly used to: (a) reproducedata obtained with real children; (b) explain brain/body mechanisms behind such data; (c) produce testable predictions.

BRAIN: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

IM-CLeVeR EU project: basal ganglia-cortex hierarchies in brain

Focus page: Further details on this research thread 
Authors: Francesco Mannella, Vincenzo Fiore, Valerio Sperati, Marco Mirolli, Gianluca Baldassarre
Topic and its relevance. We describe here research works, funded by the EU project IM-CLeVeR, directed to investigate what is the architecture and mechanisms of brain that allow primates (e.g., monkeys and children) to learn multiple skills in an cumulative fashion on the basis of intrinsic motivations. The overall architecture of brain relevant for this topic is the same as the one illustrated above in the research thread on goal-directed behaviour and habits, with the addiction of further structures important for intrinsic motivations such as the superior colliculus (important to detect changes in the environmetn caused by the organism), hippocampus (important to detect novel patterns and events), and prefrontal cortex (important to detect violation of expectations). 

BRAIN: GOALS, ACTIONS, HIERARCHIES

Brain cortico-cortical pathways and hierarchies

Focus page: Further details on this research thread 
Authors: Daniele Caligiore, Anna Borghi, Domenico Parisi, Gianluca Baldassarre
Topic and its relevance. "Embodied cognition", postulating that high-level cognition relies on the same brain mechanisms subserving sensorimotor behaviour, is a frontedge flourishing research topic of cognitive psychology and cognitive neuroscience (e.g. related to mirror neurons). Here we use computational models and theoretical analysis to propose sufficient hypotheses on the architecture, functioning, and learning processes through which the dorsal and ventral cortical pathways of brain can guide on-line action control (based on affordances and motor programs) and top-down control of them (based on context, internal motivations, and goals). 

Models of goal-directed and habitual behaviors

Focus page: Further details on this research thread
Authors: Francesco Mannella, Vincenzo Fiore, Marco Mirolli, Gianluca Baldassarre
Topic and its relevance. Organisms have a brain that evolved to produce a behaviour that enhances their survival and reproductive chances. To do this, brain produces body movements (actions) in correspondence to sensations. More sophisticated organisms, as primates, need to learn to perform and appropriately select a large number of actions depending on the environmental conditions and current internal needs. A very complex hierarchical brain architecture underlies these processes. This architecture involves the production of dynamic movementes/actions (somatosensory cortex, primary motor cortex, dorsal basal ganglia, cerebellum), their selection (premotor cortex, dorsal/medial basal ganglia), their selection and sequencing based on the organism's goals (dorsolateral prefrontal cortex, supplementary motor cortex, medial basal ganglia), their selection at a higher level based on the organisms' ultimate motivations and needs (hypothalamus, amygdala, ventral basal ganglia, orbital and ventromedial prefrontal cortex). Goal-directed behaviour involves the triggering of actions on the basis of internal motivations and goals: this typically involve amygdala, ventral/medial basal ganglia, and orbital and ventromedial prefrontal cortex. Habitual behaviour (habits) involve the automatic triggering of actions in the presence of a particular external and internal context: they typically involve dorsal basal ganglia and premotor/primary cortex. Although we have much evidence on these issues, we are still far from having a complete whole picture on these processes. The importance of this research resides in the fact that understanding these processes means understanding a large part of the whole brain functioning.

Pavlovian to instrumental transfer: models, behaviour, brain

Focus page: Further details on this research thread
Authors: Emilio Cartoni, Gianluca Baldassarre
Topic and its relevance. The study of intrumental and habitual behaviour (see above) has led us to investigate their interactions by addressing a specific psychobiology experimental paradigm called `Pavlovian Instrumental Transfer' (PIT). This issue is very important as Pavlovian and Instrumental processes are fundamental learning processes underlying adaptive behaviour. 

Basal ganglia-cortex-cerebellar system: models and applications to Parkinson

Focus page: Further details on this research thread
Authors: Daniele Caligiore, Francesco Mannella, Gianluca Baldassarre
Topic and its relevance.  The basal ganglia and cortex forms re-entrant loops through which basal ganglia contribute to select the contents of cortex. Also cerebellum forms re-entrant loops with motor cortex subserving both motor and cognitive functions. Recent evidence has also shown the existence of important cerebellum-basal ganglia anatomical bidirectional links. Basal-ganglia, cortex, and cerebellum thus form a whole system and closely cooperate to implement a large number of brain functions subserving adaptive behaviour. The specific brain mechanisms underlying these functions are only in part known. Computational models can play a key role in understanding them given the highly-dynamical complex-system nature of the basal ganglia-cortex-cerebellum system. The study of this system is also important for understanding and treating neurodegenerative diseases, such as Parkinson caused by the progressive death of dopaminergic neurons. Indeed, the system-level model-based study empowers the tracing of the effects that the Parkinson dopaminergic disregulation causes onto the whole basal ganglia-cortex-cerebellum system, thus helping to understand the multifaceted sympthoms of the diseases expressed in different patients sub-types (e.g., tremor vs. akinetic).