Laboratory of Computational Embodied Neuroscience
LOCEN mission is twofold: (a) using computational and robotic models to investigate how brain acquires behaviour, in a cumulative fashion, by interacting with the body and the environment; (b) integrating the knowledge so acquired with state-of-the-art machine learning techniques to build autonomous humanoid robots capable of learning multiple skills in an open-ended fashion (for us, ``the true artificial intelligence'').
LOCEN in brief
LOCEN history and overview.
LOCEN was funded in 2006 by Gianluca Baldassarre. Before the formation of LOCEN, various of its memebrs worked in Artificial Life: hence the importance given to embodiment/robots, adaptation, function, and complex systems. Since its birth, LOCEN has moved to developmental robotics and computational embodied neuroscience.
Currently the group is formed by 3 Researchers, 1 Senior Research Fellow, 3 PhD Students, 1 Admin/Technician, 1 Postgraduate, 1 Postdoc, various Collaborators.
LOCEN research topics
- Robots and animals: autonomous learning and development
- Robots and animals: motor control, actions
- Robots and animals: extrinsic and intrinsic motivations
- Robots and animals: goal-based learning
- Robots and animals: hierarchical reinforcement learning
- Robots and animals: attention and active vision
- Animals and robots: habitual and goal-directed behaviour and learning
- Animals and robots: classical and instrumental conditioning
- Animals: brain mechanisms of animals and humans underlying the mentioned processes (e.g. involving: amygdala, hippocampus, basal ganglia, cerebellum, associative cortex, prefrontal cortex, motor cortex, dopamine).
- Humans: Parkinson Disease, Autism, Impulsive Control Disorders
LOCEN research method
LOCEN investigates cumulative open-ended learning using two methodological approaches: (a) developmental robotics, focussed on building useful robots; (b) computational embodied neuroscience, focussed on understanding brain and behaviour of animals.
The group uses these approaches to investigate two distinct but related sets of topics and problems.
The two methods and the two sets of topics are now illustrated in detail.
LOCEN autonomous robotics
LOCEN autonomous robotics: method
The research agenda and the research bio-inspired approach of the group
The group has a research agenda that aims to build autonomous cumulative learning robots. This are robots that are capable of autonomously acquiring a number of skills in autonomous fashion, without human intervention. Although the group fully recognise the great importance of social mechanisms for the acquisition of complex behaviours like those acquired by primates (e.g., social enhancement, joint attention, imitation, language, teaching, etc.), the group focusses its research on individual learning processes so to have a higher impact in this study. In this respect, the ``holy grail'' of LOCEN is to arrive to build robots that, like children, are capable of acquiring an increasingly sophisticated repertoire of sensorimotor and cognitive skills, from simple to complex ones, in a cumulative fashion, without external intervention. For this reason, we follow the developmental robotics approach to build its robots.
In this respect, the group believes that fundamental breakthroughs in specific fields of robotics and machine learning will come from the study of biological systems. The reason is that organisms are the result of a run of the ``gigantic'' genetic algorithm represented by natural evolution: such algorithm has found solutions to some problems that engineering approaches will not beat for long. The idea is so to reverse-engineer brain and behaviour of real animals to propose radically new ideas to autonomous robotics and machine learning, that might synergies with those discovered with more traditionalapproaches.
LOCEN autonomous robotics: topics
The topics investigated by the group are all directed to support its ``mission'' of building autonomous, cumulative learning robots. We believe that four critical challenges need to be overcome to accomplish this goal:
Dynamic behaviour, compliant hardware and muscles
All that ultimately counts, for both robots and animals, is action. Action means a change of the sensorimotor body in space that in turn can change the environment (e.g., to manipulate objects) and the body-environment relation (e.g., to look at something, or to change position in space). Behaviour emerges from the dynamic interplay between the robot's action, that changes the body-environment relation, and the close-loop physical and perceptive feedback flow that results from this. Part of the extraordinary capabilities that animals display in these interactions rely upon the hardware of their body, in particular their compliant skeleto-muscular system. For this reason, one topic of interest of the group is the challengs and opportunities offered by robots endowed with a biomorphic compliant hardware (e.g., bio-materials, biologically-plausible structure, pneumatic actuators).
Active vision, proprioception and abstraction
Extrinsic and intrinsic motivations, and learning
As the group is interested in autonomous cumulative learning, he is profoundly interested on the mechanisms that can guide learning in an open-ended fashion without external intervention. Motivations are a key element for this as they can: (a) drive the robots to perform different behaviours in different conditions; (b) generate the learning signals needed to guide their learning. Two types of motivations are studied by the group. The first are extrinsic motivations, i.e. motivations related to the task assigned to the robot (what is the best reward function function?). The second are intrinsic motivations. These are a rather novel concept in autonomous robotics and the group is keenly investigating them. Intrinsic motivations drive behaviour and generate learning signals on the basis of the fact that the controller is indeed acquiring knowledge with a high rate, e.g. is forming good abstractions, is learning to predict well, or is acquiring the sensorimotor competence to accomplish a goal. Intrinsic motivations are fundamental for autonomous robots as allows them to acquired knowledge and skills in the absence of tasks from the user, so that when some tasks are assigned to them they can exploite the knowledge acquired autonomously to readily solve them. The pivot of all these processes is of course learning, so the group keenly investigates all forms of learning useful for robots, from associative to unsupervised learning, from supervised to reinforcement learning (the latter is central in several models of the group).
A video showing a humanoid robot (iCub) that explores a `mechatronic board' under the drive of intrinsic motivations, and autonomously discovers and learns that pressing some buttons turns on some lights. In a later stage, the robot exploits the acquired action-outcome contingencies to accomplish useful purposes. For more detailed videos see here and here: these videos are explained in the web-pages here and here; the system is explained in detail in the article here. Work carried out within the project IM-CLeVeR.
The cumulative acquisition of skills and cognitive capabilities requires that the newly acquired knowledge: (a) is stored without destroying previously acquired knowledge; (b) supports the acquisition of further knowledge. The first goal is achieved through hierarchical architectures that avoid that the acquired different pieces of knowledge interfere with each other. The second goal is accomplished by investigating how the acquired knowledge can be transferred to new tasks to be solved (e.g., as in transfer reinforcement learning). Because of these two goals, the group is keenly interested in developing hierarchical reinforcement learning architectures.
Video of a humanoid robot (iCub) learning to throw a ball to a target based on a hierarchical reinforcement learning system with sophisticated generalisation capabilities (generation of new dynamic movement primitives on the fly based on the similarity of the new goal with respect to previously acquired goals). This work was carried out in collaboration with Bruno Castro da Silva and Andrew Barto, from the University of Amherst Massachusetts.
LOCEN brain and behaviour
LOCEN brain and behaviour: method
Computational Embodied Neuroscience
The group has developed an original research method to study brain and behaviour, rooted on system-level computational neuroscience and artificial life, named Computational Embodied Neuroscience (CEN).
Differently from other computational neuroscience approaches, CEN aims to understand the brain with a ``top-down approach'' starting from behaviour and function. The key idea behind this is that the brain evolved to allow animals to act so as to improve their survival and reproductive chances. So to fully understand how brain works, we need to understand not only its mechanisms (anatomy and physiology, the common focus of neuroscience) but also its functions, i.e. ``what it is for''. This is why the group is keenly interested in linking the knowledge produced not only by neuroscience but also by psychology and psychobiology.
From system-level to detailed models (multi-scale models)
Another consequence of the goal of CEN of studying whole systems capable of acting is the tendency to build system-level models, reproducing the macro-architecture of various cortical and sub-cortical brain areas that underlie the target behaviours. This is in fact needed if one wants to understand how a certain area of brain works not studying it in isolation, but how its inner mechanisms play a certain function within a whole system. Of course, often the aspect of the system which are not under focus are represented in an abstract fashion, but they are nevertheless there. After sedimenting knowledge at the system-level, the group usually starts to refine the micro-architecture and functioning of the various components of the model (usually focussing on a subset of them). This leads to have multi-scale models that encompass not only the system-level but also the meso-level (e.g., different nuclei of basal ganglia) and micro-level (e.g., the canonical architecture of cortex) of brain. For these overall purposes, our models are usually based on firing rate neurons and leaky neurons, and only recently on spiking neurons (integrate-and-fire neurons). Recently, the group has started to build probabilistic graphical models whose functioning is implmemented on the basis of Bayesian inferences. These models have the advantage of facilitating system-level modelling at a high (often functional) level before moving to more detailed neural models, and also of offering the possibility to directly evaluate the goodness of models against empirical data, for example from brain imaging. We are also interested in probabilistic interpretations of brain processing (from which our recent interest for spiking neurons).
Aiming to build cumulative models
The hallmark of science is cumulativity. Too often different computational models are build to interpret different experiments. Instead, CEN aims to build models that allow the explanation of an increasing number of specific brain and behavioural data. This allows the isolation of general principles, the integrated theoretical systematisations of whole classes of phenomena, and so to help psychology and neuroscience to overcome the polverisation of results and views that they often encounter for their need of focussing. Integrated theories and models allow the production of detailed hypotheses that fill in the knowledge gaps of psychology and neuroscience and produce specific empirical predictions that can be tested in new empirical experiments.
The importance of a close dialogue between models and empirical data
Differently from other computational approaches to the study of brain and behaviour, CEN stresses the importance of having a tight, continuous dialogue with specific empirical data from psychology and neuroscience. The idea is that the understanding of brain and behaviour should proceed as any good science, namely it should rest on two pillars: (a) the theoretical understanding of the investigated phenomena, based on formal computational models; (b) the empirical investigation of such phenomena to select the best theories, models, and predictions. In this respect, we see computational modelling as a powerful theoretical means that should guide empirical research on brain and behaviour. The ultimate proof that computational modelling of brain and behaviour has successfully accomplished its mission is its capacity to change the daily research of the empirical neuroscientist and psychologist, and to publish papers in top journals of neuroscience and psychology.
The importance of embodiment: sensorimotor and visceral
We believe that brain generates behaviour by dynamically interacting with the environment through sensors and actuators in a circular fashion (embodiment). Sensors furnish a rich, redundant and noisy amount of information to organisms. Actuators are in turn noisy, redundant, compliant. Before facing high-level cognition problems, brain has to solve the problems posed by such input/output information channels (and also exploit the opportunities they offer) . The resulting computations might be radically different from those that would stem from, say, a clean, symbolic type of input/output information. For these reasons, we think that good models of brain and behaviour should function in simulated or real robotic systems that have the same sensors and actuators as the investigated animals. This poses strong challenges to models, especially because neuroscientists and psychologists often overlook them and require us to focus ``on their problems'', concerning higher-level aspects of cognition. For this latter reason, the more biologically constrained models produced by the group so far often use localistic representations, abstract input/output information codes, simplified environments. However, the group is fully aware of the importance of scaling up models to more realistic input/output and environmental conditions, so we try our best to incorporate in the models critical elements of a true embodiement, e.g. the sensorimotor loop and the test with simulated/real robots. Aside this, the group gives also a lot of important to a second type of ``embodiment'', most of the times neglected but as important as ``sensorimotor embodiment''. This might be called ``visceral embodiment'' and refers to the key relation that the brain has with the visceral body and its homeostatic regulations. These regulations are at the basis of extrinsic motivations and the subjective value (biologically saliency) that organisms assign to objects and experiences.
The importance of learning
We believe that for a large part the brain structure is as it is because it has not only to express behaviour but also to learn it. For this reason, most of our models aim not only to reproduce target behaviours, and the neural machinery to do so, but also the learning processes that lead to its acquisition with experience, and hence the physiological processes underlying this. For this reason we are keenly interested in studying all forms of biologically plausible learning: Hebbian learning, differential Hebbian learning, competitive self-organised learning, and reinforcement learning, and goal-based learning.
LOCEN brain and behaviour: topics
Given the interest of the group in understanding how brain and behaviour supports cumulative learning in organisms, our research is focussed on the following topics:
Vision and active vision (superior colliculus, brain dorsal and ventral pathways, basal ganglia, cortex)
The bio-contrained models developed by the group focus on two types of perception: proprioception and vision. Often we use proprioception in our models to guide low-level behaviour, but we do not study it per se. We instead are very interesting in studying vision, and especially active vision. The reason is that vision is the primary information source for primates and has a paramount role in guiding action (via the brain dorsal pathway, involving parietal and premotor cortex) and to support higher-level cognition, such as decision making and planning (via the ventral pathway, involving inferotemporal cortex and prefrontal cortex). We study vision not as a passive source of information but rather as an active one. In particular, we are interested in studying how overt attention, and the high resolution of the fovea with respect to peripheral vision, can actively search and gain information in the environment, and ignore irrelevant one, on the basis of the animal's goals. We are thus interested in bottom-up attentional processes, that drive eye gaze on most informative parts of the environment, and in top-down attentional processes, that drive the eye to collect information based on goals; and we are of course interested in their rich interplay. We also believe that attention is pivotal for the rest of behaviour and indeed there is a strong coupling between vision and arm/hand manipulation actions, with attention representing a powerful guidance for controlling such actions via the selection of suitable inputs (``eyes lead, arms execute'').
Extrinsic motivations, Pavlovian and instrumental learning (amygdala, nucleus accumbens, dopamine)
Behaviour needs to be driven. Learning needs to be guided. Extrinsic motivations are expressed by parts of brain (amygdala, nucleus accumbens, dopaminergic and other neuromoduatory systems) at the interface between visceral body and cognitive processes. We are interested in studying how extrinsic motivations drive behaviour, directing it to specific activities, or how they generate learning signals that guide learning processes. In this respect, we are very interested in investigating how areas such as the amygdala can perform Pavlovian associations that allow triggering important internal reactions (e.g., for the internal regulation of visceral body and the neuromodulation of brain) and external behavioural reactions (feeding, approaching, orienting) in correspondence to biologically salient stimuli (e.g., food, water, sex) or stimuli anticipating them (conditioned stimuli). Also, we are very interested in understanding instrumental behaviour, i.e. the processes that allow organisms to learn to trigger (learned, instrumental) behaviours, when particular conditions are present, if this leads to rewards (S-R behaviour).
Intrinsic motivations (superior colliculus, hippocampus, dopamine, noradrenaline)
Aside extrinsic motivations, we are interested in studying intrinsic motivations, i.e. the motivations at the core of the performance and acquisition of actions ``for their own sake'', i.e. not for the achievement of results that directly increases biological fitness (e.g., food or money). Intrinsic motivation systems have evolved as they drive exploration and learning in the absence of extrinsic outcomes, and have the function of leading animals to acquire knowledge and skills that will be readily usable in the future when such extrinsic outcomes become available.
Hierachical sensorimotor brain, habitual behaviour (hierarchical cortex, multiple basal ganglia loops, cerebellum)
Organisms' cumulative learning of sensorimotor behaviour and higher-level cognition requires a hierarchical soft-modular brain architecture that links motor behaviour to perception at different levels of abstractions and coupling. This architecture is organised in at least three levels, investigated by the group with system-level models. (1) At the lowest level, the close-loop between somatosensory cortex and primary motor cortex (forming a loop with sensorimotor basal ganglia involving putamen/globus pallidum) and involving cerebelllum, implements the dynamic production of movements (e.g., a reach, a grasp); (2) the dorsal brain pathway, encompassing visual cortex, parietal cortex (encoding affordances), and premotor cortex (encoding repertoires of actions) (which forms a loop with sensorimotor basal ganglia involving putamen-caudatum/pallidum-subtantia nigra reticulata), implement the on-line control of action (e.g., to guide a reach to an object, or to shape the hand to grasp it); (3) the ventral brain pathway, encompassing the visual cortex, the inferotemporal cortex (for object recognition), and the prefrontal cortex (for working memory and multimodal sensory integration) (which forms a loop with associative basal ganglia involving caudatum/pallidum), implements high-level decision making and executive control of behaviour; (4) the cortex communicating with sub-cortical limbic structures, such as amygdala and hippocampus (which forms a loop with limbic basal ganglia involving nucleus accumbens), processes value and encodes biologically salient action-outcomes and goals (e.g., food). The group studies this complex hierarchical system by usually focussing on different parts of it but always keeping in mind the overall architecture.
Goals, goal-directed behaviour, decision making (amygdala, hippocampus, nucleus accumbens, prefrontal cortex)
Starting form sensorimotor behaviour, the group is now gaining knowledge and skills for the investigation of higher-level cognition, in particular in relation to goal-directed behaviour and decision making (levels 3 and 4 of the framework of the previous point). ``Goals'' are now becoming a critical concept for the group for the pivotal role they play in cognition and, in particular, in autonomous cumulative learning. A goal is an internal representation of a future world state that is activated internally and drives action (and learning) for its accomplishment. The current investigation of ``goal-directed behaviour'' and decision making, strongly supported by theory-driven model-free and models-based reinforcement learning, is one of the hottest fields of investigation of cognitive science. Goals also play a key role in autonomous cumulative learning: a critical function that intrinsic motivations play is the self-generation of goals. If you look with attention a little child at play, you will realise how her/his autonomous exploration and learning is strongly guided by an incessant autonomous setting, pursuit, accomplishment, monitoring, and switch of goals. Goal-based processes, greatly enhanced by the uniquely-developed human prefrontal cortex, focus attention, inform on success, drive learning, and organise action in ways that render autonomous learning ``explosive'', i.e. powerful, omni-directional and open-ended.
The presence in the same group of different perspectives and methods (robotics, modelling, neuroscience and psychology) on the same topics give the group a number of advantages:
- An approach to problems of autonomous robotics and cognitive-science that is profoundly interdisciplinary
- The capacity to coordinate and play a key role in large robotics/cognitive-science projects involving different approaches, themes, and teams
- An ``eagle-eye view'' on robotics and cognitive-science issues that allows the group to see phenomena, problems, and solutions overlooked by other more focussed, but possible near-sighted, views and approaches.
Focus Web-Pages on LOCEN Research Threads
Below we report a table describing the main research threads of LOCEN, organised by two main themes (Hierachies, actions; open-ended learning, intrinsic motivations) and main focus (brain, behaviour, robots, applications: but several threads are interdisciplinary and involve more than one of these).
Below the table we explain each research thread in some detail: click the title of each thread to go to a focus web-page dedicated to the thread and containing more explanations and material on them (we are working on them to add detailed explanations, pictures, videos, list relevant publications, etc. ).
The references and pdf files of publications related to the research threads can be retrieved from this web-site here:
Publications of LOCEN, including pdf files, by year.
Open ended learning,
Hierarchical sensorimotor architectures, and extrinsic motivations, in robots and animal brain
Modular reinforcement-learning controllers for robots to learn multiple skills (by Daniele Caligiore, Paolo Tommasino, Annalisa Ciancio, Valentina Meola, Gianluca Baldassarre).
Topic and its relevance. Building architectures that allow robots to learn multiple sensorimotor skills, possibly transferring knowledge between them, is a central open challenge for autonomous robotics. In particular, it is paramount to produce autonomous cumulative learning robots. This is also important to suggest possible architectures and processes through which brain solves the same problems.
Models of the reaching skills, and of their development, in children (by Daniele Caligiore, Domenico Parisi, Gianluca Baldassarre).
Topic and its relevance. Reaching, i.e. the capacity to get own hands in contact with objects in the environment, is a fundamental motor skill for primates and humans, at the basis of their capacity to interact with, manipulate, and change the world at own benefit. Here we use computational models to understand the mechanisms underlying learning and development of such reaching skill. The study of reaching is supported by the availability of a wealth of empirical data against which models can be tested. The model is mainly used to: (a) reproduce data obtained with real children; (b) explain brain/body mechanisms behind such data; (c) produce testable predictions.
Bio-constrained models of affordances and brain cortico-cortical pathways (by Daniele Caligiore, Anna Borghi, Domenico Parisi, Gianluca Baldassarre).
Topic and its relevance. "Embodied cognition", postulating that high-level cognition relies on the same brain mechanisms subserving sensorimotor behaviour, is a frontedge flourishing research topic of cognitive psychology and cognitive neuroscience (e.g. related to mirror neurons). Here we use computational models and theoretical analysis to propose sufficient hypotheses on the architecture, functioning, and learning processes through which the dorsal and ventral cortical pathways of brain can guide on-line action control (based on affordances and motor programs) and top-down control of them (based on context, internal motivations, and goals).
The video above shows the performance of the iCub robot during a mental rotation task. A bio-inspired model of dorsal and ventral interplay controls the iCub behaviour in order to accomplish the mental rotation task. If during mental rotation the robot turns its wrist in the same direction as the mental rotation the robot performs better the task. By contrast, the performance is worse if the robot turns its wrist in the opposite direction to the mental rotation.
Emobodied models of goal-directed behaviour and habits (amygdala, ventral/medial/dorsal basal ganglia, prefrontal cortex, motor cortex) (by Francesco Mannella, Vincenzo Fiore, Marco Mirolli, Gianluca Baldassarre)
Topic and its relevance. Organisms have a brain that evolved to produce a behaviour that enhances their survival and reproductive chances. To do this, brain produces body movements (actions) in correspondence to sensations. More sophisticated organisms, as primates, need to learn to perform and appropriately select a large number of actions depending on the environmental conditions and current internal needs. A very complex hierarchical brain architecture underlies these processes. This architecture involves the production of dynamic movementes/actions (somatosensory cortex, primary motor cortex, dorsal basal ganglia, cerebellum), their selection (premotor cortex, dorsal/medial basal ganglia), their selection and sequencing based on the organism's goals (dorsolateral prefrontal cortex, supplementary motor cortex, medial basal ganglia), their selection at a higher level based on the organisms' ultimate motivations and needs (hypothalamus, amygdala, ventral basal ganglia, orbital and ventromedial prefrontal cortex). Habits involve the automatic triggering of actions in the presence of a particular external and internal context: they typically involve dorsal basal ganglia and premotor/primary cortex. Goal-directed behaviour involves the triggering of actions on the basis of internal motivations and goals: this typically involve amygdala, ventral/medial basal ganglia, and orbital and ventromedial prefrontal cortex. Although we have much evidence on these issues, we are still far from having a complete whole picture on these processes. The importance of this research resides in the fact that understanding these processes means understanding a large part of the whole brain functioning.
Questions and goals. What are the functions and mechanisms (i.e., architectural, functioning, and learning features) underlying goal-directed behaviour and habits? How are dynamic movement primitives and goals formed? How do habits form? How is goal-directed behaviour controlled by internal motivations and external context? What are the mechanisms of arbitration between goal-directed and habitual behaviour?
Methods. We investigate these issues through bio-constrained computational models. We start with the overall goal of accounting for specific psychbiology experiments, usually carried out with rats/mice, for example on: classical conditioning, second order conditioning, stress coping, instrumental conditioning, devaluation, Pavlovian to intrumental transfer, etc. We constrain the mechanisms of models on the basis of the known anatomy of brain and lesion experiments: we start from the macro-anatomy and then, when made necessary by the targeted behaviours and the refinement of the model, we specify the meso architecture (e.g., internal modules of basal ganglia, amygdala; specific areas of ventro-medial prefrontal cortex) and micro architecture (micro-anatomy and cell types of amygdala, basal ganglia, and cortex).
Results. This approach allows us to isolate sufficient hypotheses to account for the target behaviours and the related neuroscientific evidence in a coherent and comprehensive way. The high number of constraints imposed on the models at the behavioural and neuroscientific anatomical/functioning/learning levels also increase the ``probability of the necessity'' of the proposed hypotheses, i.e. that the models we propose capture what actually happens in brain. The models so proposed produce specific empirical predictions that can be tested with further empirical experiments: we have started collaborations with neuroscientists to test some predictions generated by our models.
Conclusions. Based on this integrated approach, we have formulated models that incorporate sufficient hythesis on the brain mechanisms that might underlies various behaviours studied in psychobiological experiments, e.g. various forms of classical conditioning, devaluation, stress coping experiments. These models produce predictions, and furnish comprehensive pictures often lacking in psyhchobiology and neuroscience, that foster and guide further empirical research on the issues.
Models and human experiments of Pavlovian instrumental transfer; amygdala, accumbens and prefrontal cortex (by Emilio Cartoni, Gianluca Baldassarre).
Topic and its relevance. The study of intrumental and habitual behaviour (see above) has led us to investigate their interactions by addressing a specific psychobiology experimental paradigm called `Pavlovian Instrumental Transfer' (PIT). This issue is very important as Pavlovian and Instrumental processes are fundamental learning processes underlying adaptive behaviour.
Embodied dynamical models of the basal ganglia-cortical system based on echo-state networks (for cortex) and channel-based models (for basal-ganglia) (by Francesco Mannella, Gianluca Baldassarre).
Topic and its relevance. This research thread, started recently, concerns the study of cortex viewed as a dynamical system whose dynamics is regulated by basal ganglia. The importance of this reseach resides in the fact that the basal-ganglia and cortex form segregated loops that are a fundamental building block underlying multiple cognitive processes, from associative sensory processing to motor behaviour, thinking, plannig, and reasoning.
Questions and goals. How does cortex encode and process information, e.g. to control motor behaviour? How do basal-ganglia modulate such processing? What are the plasticity process that characterise this system?
Methods. We model cortex with echo-state networks and basal ganglia with re-entrant localistic neural networks. The models use associative and reinforcement biologically plausible learning rules.
Results. The models show how cortex migth process information in a highly dynamical fashion, and how basal-ganglia might select the contents within cortex by modulating its dynamics.
Conclusions. The basal-ganglia and cortex form a highly integrated system with an inherently dynamical nature. A basic functioning and learning template can sub-serve multiple cognitive processes.
Bio-constrained models of the basal ganglia-cortex-cerebellum system: applications to the study of Parkinson and other diseases (by Daniele Caligiore, Francesco Mannella, Gianluca Baldassarre).
Topic and its relevance. The studies of the basal-ganglia cortical systems allow to model and investigate the ways in which particular neurodegenerative diseases, such as Parkinson caused by the progressive death of dopaminergic neurons, produce different sympthoms like tremor and akinesia.
Cumulative open-ended learning of multiple skills, driven by intrinsic motivations, in robots and animal brain
Theory and empirical experiments on intrinsic motivations (by Vieri Santucci, Daniele Caligiore, Magda Mustile, Marco Mirolli, Gianluca Baldassarre).
Topic and its relevance. Intrinsic motivations (IMs) are related to curisity, exploration, the interest for novel objects and surprising evens, and the drive to learn motor skill. IMs operate in the absence of a direct biological pressure and feedback (as in the case of extrinsic motivations, i.e. the classic motivations related to homeostatic regulations and survival). IMs are a fundametnal topic of investigation as they play a key role in human well being, art, science, and technology. They are also important for autonomous robotics as they allow the construction of cumulative learning robots.
Project IM-CLeVeR: Bio-constrained models of intrinically-motivated cumulative learning in robots and organisms (by Francesco Mannella, Vincenzo Fiore, Valerio Sperati, Marco Mirolli, Gianluca Baldassarre).
Topic and its relevance. We describe here research works, funded by the EU project IM-CLeVeR, directed to investigate what is the architecture and mechanisms of brain that allow primates (e.g., monkeys and children) to learn multiple skills in an cumulative fashion on the basis of intrinsic motivations. The overall architecture of brain relevant for this topic is the same as the one illustrated above in the research thread on goal-directed behaviour and habits, with the addiction of further structures important for intrinsic motivations such as the superior colliculus (important to detect changes in the environmetn caused by the organism), hippocampus (important to detect novel patterns and events), and prefrontal cortex (important to detect violation of expectations).
Questions and goals. What is the architecture of brain that underlies the cumulative acquisition of multiple skills? What are the intrinsic motivation mechanisms that drive primates to acquire goals and skills in a cumulative fashion? How do extrinsic motivation allow primates to exploit the goals and skills aquired with intrinsic motivations?
Methods. We built bio-constrained models reproducing these aspects of brain: (a) the hierachical organisation of skills based on basal ganglia-cortical loops and cortico-cortical pathways; (b) the encoding of habitual skills and goal-directed behaviour within such architecture; (c) the intrinsic motivation (IM) systems guiding cumulative learning, in particular based on `novelty-based IMs' (hipocampus) and `prediction-based IMs' (superior colliculus, cortex) generating dopamine-based learning and motivational signals; (d) the extrinsic motivations, e.g. to get food, relying on amygdala and again on the dopamine system (ventral tegmental area and substantia nigra pars compacta). The models are bio-constrained in the sense that their macro-architecture and macro-functions reflect those of brain. The models are tested within simulated humanoid robots (iCub) reproducing the behaviours observed in the `Mechatronic Board Experiment', an experiment carried out with monkeys and children freely interacting with a `mechatronic board' to acquire skills on the basis of IMs (the mechatronic board, built in IM-CLeVeR, has manipulanda, buttons, lights switching on and off, opening boxes, and sounds).
Results. The result of the research is the proposal of the possible components of brain, and their connections and learning and functioning mechanisms, sufficient to produce IM-based acquisition of skills.
Conclusions. To our knowledge, the proposed models represent the first sufficient hypotheses on how brain allows primates to acquire skills on the basis of intrinsic motivations and later recall them on the basis of extrinsic motivations.
Robotic models of multiple skill learning driven by intrinsic motivations (by Vieri Santucci, Marco Mirolli, Gianluca Baldassarre).
Topic and its relevance. Developing robots able to autonomously discover, select, and solve multiple new tasks in a cumulative open-endeed fashion is an important issue for autonomous robotics. It becomes even crucial if we want to build robots capable of solving multiple problems in real environments posing challenges that are unknown at design time. There are two key `ingredients' necessary to build these kind of robots. The first are intrinsic motivations (IMs): these can drive autonomous learning of robots in an open-ended fashion in the absence of tasks assigned to the robots by the users. The second are hierarchical architectures: these are needed to store multiple skills, drive their acquistion with IMs, learn goals related to skills, and form complex skills based on simpler skills.
Ecological active vision and intrinsic motivations: robotic and embodied models; models of superior colliculus and basal ganglia (by Dimitri Ognibene, Valerio Sperati, Rodolfo Marraffa, Gianluca Baldassarre).
Topic and its relevance. This project (initially funded by the EU project MindRACES and later by the EU project IM-CLeVeR) is on LOCEN's approach to vision called `Ecological Active vision - EAV'. EAV is grounded on the `active vision' approach, based on an actively-moved small fovea plus a low-resolution periphery, augmented with four principles: (a) a strong coupling of bottom-up and top-down attention processes; (b) the use of reinforcement-learning to acquire top-down attention skills; (c) the use of attention and vision to support pragmatic action (e.g., reaching and grasping) rather than vision per-se, in particular a strong spatial coupling between attention and manipulation actions; (d) the use of a novel Potential Action Memory component to collect information on the best places to visit with the fovea. Lately we have linked EAV with intrinsic motivations (IMs), in particular IM related to the perception of movement in the world and agency (i.e., the agent's perception of the capacity to cause movement in the world with own actions).
+Me": A new wearable interactive device to support and develop communication skills in children with autism. See also dedicated web-page here. (by Beste Ozcam, Valerio sperati, Daniele Caligiore, Gianluca Baldassarre)
Topic and its relevance. Autism (or Autism Spectrum Disorder - ASD) is a very important mental disorder affecting 1 out of 60 people in the world. The most impariging sympthom of autism is the difficulty to communicate with others. This project is direced to build an interactive wearable device that supports and motivates social interaction, and contributes to develop communication skills, in children with autism.
Questions and goals. Which sensors and interfaces should the device have to best support and motivate autistic children, characterised by a peculiar sensation processing, to communicate with the outer world? Which sensors and interfaces should the device have to allow caregivers to best engage and communicate with autistic children? Which tests can be carried out to evaluate, select and develop the features of the wearable?
Methods. The wearable is a soft-touching pillow endowed with sensors (e.g., touch, pressure, michrophones, physiological detectors for heart-bit and other body features), actuators (e.g., sounds and lights) and, in the future, a tablet wi-fi inteface. The wearable feaures are tested with autistic children, together with specialised therapists and parents, to best develop its features.
Results. At the moment we have realised the prototype of the device on which we have collected very positive preliminary feedback from caregives and parents. We are now preparing systematic tests to measure the effects of the wearable device features with children.
Conclusions. The wearable represents a higly-innovative device with a notatble potential to support and motivate child-caregiver and child-parent interactions based on the peculiarities of the autistic child cognition. Empirical tests are needed to validate such potential.
Robotic models of collective cooperative behaviours in navigation tasks (by Gianluca Baldassarre, Domenico Parisi, Stefano Nolfi).
Topic and its relevance. This is research thread, now terminated, was conducted within the EU funded project Swarm-bots. Collective robotics involves the use of multiple robots to carry out tasks that could not be carried out by single robots alone. For some tasks, the simplicity of single robots in terms of sensors, actuators, and communication capabilities can give robustness and low-cost to the whole ``swarm'' of robots. In this cases, the coordination between robots, needed to carry out a common task in cooperation, can rely on distributed (vs. centralised/hierarchical) coordination and communication mechanisms typically exploited by social insects (e.g., ants and bees), for example stigmergy (what one robot does with its body and in the environment is directly exploited by the others for coordination).
Questions and goals. What minimal sensors and actuators can robots have to best coordinate in tasks involving colletive navigation on rough terrains? How can their controllers be organised to give rise to an effective distributed emergent coordination? What are the capabilities and properties of the emergent behaviours?
Methods. The project Swarm-bots built relatively simple natigating robots (about 15 cm of diameter) that can navigate on rough terrains with two tracks and have a turning turret that can rotate on the tracks (as a tank) and endowed with a gripper with which the robots can be attached to the turret of other robots. Here we exploited a special sensor, suggested and developped by the coordination opportunities revealed by the simulations, located between the turret and the truks and that can sense the direction and intensity with which a robot is pulled by other robots attached to its turret. We used these robots, both simulated and real, to form groups of robots permanently attached between them (4 to 8). The robot groups were tested in tasks where they had to freely explore a (possibly rough) open field, or mazes in search of a light target. The controller was a simple neural network getting as input the direction of pulling of the companion robots and returning as output the speed of motion of the two robot's tracks. The controller's parameters (connection weights of the network) were evolved with a genetic algorithm using as fitness the speed of motion of the whole group in any direction, and, when present, towards the light target. Importantly, the individual of the genetic algorithm was one robot controller that was then copied without changes into all the robots of a robot team: this mimics the similarity of DNA of the members of a colony of social insects (``group selection'').
Results. The resuls show that the genetic algorithm leads to the emergence of a controller for which the robots tend to move fast when pulled from behind by others, but also to turn left or right when they are pulled from the left or the right hand side (conformist behaviour). This behaviour gives rise to a fast coordination between the robots so that they rapidly agree on a common emergent direction of motion. This behaviour also allows the group to change direction of motion when they encounter an obstacle, and can easily be biased by the perception of a litght-target to move to it. Results also show that the emerged coordination mechanism leads to an suddent abrupt increase of coordination: a measure of the entropy of the group direction reveals how this resembles a phase transition typical of collective physical systems.
Conclusions. The research has shown how simple robots endowed with simple controllers can acquire, by evolution, very effective and sophisticated behaviours if these can rely upon stigmergic sensors and mechanisms exploiting self-organisation.
- Vincenzo Fiore (PhD/Postdoc) (October 2008 - October 2012)
- Paolo Tommasino (Research Fellow) (June 2011- July 2012)
- Fabian Chersi (Postdoc) (June 2009 - April 2011)
- Massimiliano Schembri (Research Fellow) (April 2006 - December 2009)
- Stefano Zappacosta (Research fellow) (April 2006 - June 2009)
- Dimitri Ognibene (PhD/Postdoc) (March 2006 - April 2009)
- Tomassino Ferrauto (Research Fellow) (January 2006 - February 2009)
- Angelo Rega (Research Fellow) (March 2006 - June 2006)
PHD/MA research supervised by LOCEN
|Caligiore D., Mustile M., Cipriani D., Redgrave P., Triesch J., De Marsico M., Badassarre G. Intrinsic motivations driving learning of eye movements: an experiment with human adults. In: PLoS One, vol. 10 (3) article n. e0118705. Public Library of Science, 2015.|
|Ognibene D., Baldassarre G. Ecological active vision: four bio-inspired principles to integrate bottom-up and adaptive top-down attention tested with a simple camera-arm robot. In: IEEE Transactions on Autonomous Mental Development, vol. 7 (1) pp. 3 - 25. IEEE, 2015.|
NOTE: Some sections below are under construction!
LOCEN in the Web
Joining LOCEN, or doing a research thesis with it
Below we list links to our web-pages giving resources on work-tools used by LOCEN
Gianluca Baldassarre, Ph.D.,
Laboratory of Computational Embodied Neuroscience (LOCEN),
Istituto di Scienze e Tecnologie della Cognizione (ISTC),
Consiglio Nazionale delle Ricerche (CNR),
Via San Martino della Battaglia 44, I-00185 Roma, Italy
Tel: +39 06 44 595 231
Fax: +39 06 44 595 243
How to get to the ISTC-CNR: directions, maps, hotels
From the airport “Roma Fiumicino”
This is the main airport of Rome (see map below). Follow the indications for the train terminal inside the airport. At the train terminal, buy the tickets at the ticket office for the train “Leonardo Express” to “Roma Termini” central station, where you have to get off (last stop). The ticket can be purchased at the ticket office, at the news agents and at the automatic ticket machines (cost: 11 euros). The train runs from 6.30 am to 11.30 pm, and departs every 30 minutes. Once at Termini Station, you can reach ISTC-CNR on foot (10 minutes, see map below) or take the subway. To take the subway, look for signs of subway B, Rebibbia direction (“Metro B, direzione Rebibbia”; Rome has only two subway lines, A and B.). Buy tickets from the automatic ticket machines (cost: 1 euro). Get off after one stop at subway stop “Castro Pretorio”. ISTC-CNR, a brown historical two-floor building, is round the corner of this subway stop at the beginning of Via S. Martino della Battaglia, at the entrance number 44 (see map below).
If you want to spend less money for the train, at Fiumicino airport take the train to “Roma Tiburtina”, where you get off (the trip takes about 45 minutes as this is a local train; this is usually not the last stop). At the train terminal, you can buy the ticket for this train (cost: 4.5 euros). Once at Tiburtina Station (the second biggest station in Rome), take the subway B, Laurentina direction (“Metro B, direzione Laurentina”). Get off at subway stop “Castro Pretorio”. As explained above, ISTC-CNR is round the corner of this subway stop.
The taxi from Fiumicino airport to Rome costs about 40 euros (supplements might be asked for luggage, night-time runs and public holidays) and the trip takes approximately 45 minutes.
From the airport “Roma Ciampino”
This is the second airport of Rome, and is very small. It is closer to Rome than Fiumicino airport (see maps below). Get out of the airport terminal and ask the bus drivers standing outside, near the buses you see once out, about a bus that takes you to Anagnina Subway Station (“Stazione della Metropolitana Anagnina”). The subway of Anagnina Station is on the subway line A. Once at Anagnina Station, buy a ticket from the automatic ticket machines or the news agents (cost: 1 euro). At Anagnina Station, take the subway A to Roma Termini Station. Once at Termini Station, you can reach ISTC-CNR on foot (10 minutes away, see map below) or take the subway. Otherwise, switch subway line to subway B following the signs for “Metro B, direzione Rebibbia”. Get off after one stop at subway stop “Castro Pretorio”. ISTC-CNR, a brown historical two-floor building, is round the corner of this subway stop at the beginning of Via S. Martino della Battaglia, at the entrance number 44 (see map below).
Taxis from Ciampino airport to ISTC-CNR charge about 30 euros, and take about 20 minutes to get there.
Rome has two airports, "Fiumicino" and "Ciampino". ISTC-CNR is situated at the centre of Rome:
ISTC-CNR is at "Via San Martino della Battaglia 44", one subway stop from central Termini Station (subway B, stop Castro Pretorio):
Hotels close to ISTC-CNR
Hotel Villafranca (4 stars), Via Villafranca 9, tel: +39-06-4440364
Champagne Hotel (4 stars), Via Vittorio Bachelet 4, tel: +39-06 -927209 or +39-06-492721
Artdeco Hotel (4 stars), Via Palestro 19, tel: +39-06-4457588
Hotel S. Marco (3 stars), Via Villafranca 1, tel: +39-06-490437
Hotel Piemonte (3 stars), Via Vicenza 32/a, tel: +39-06-4452240
Hotel Montecarlo (3 stars), Via Palestro 17/a, tel: +39-06-4460000
Hotel Astoria, (3 stars), Via Vittorio Bachelet 8, tel: +39-06-4469908
Hotel Lux (3 stars), Via Gaeta 14, tel: +39-06-4441692
Hotel Brasile (3 stars), Via Palestro 13, tel: +39-06-4819486
Hotel Villa delle Rose srl (3 stars), Via Vicenza 5, tel: +39-06-4451795
Hotel Dolomiti - Sada sas (3 stars), Via San Martino della Battaglia 11, tel: +39-06-491058 or +39-06-4957256
Hotel Fiamma (3 stars), Via Gaeta 61, tel: +39-06-4818436 or +39-06-4818912
Hotel Siviglia (3 stars), Via Gaeta 12, tel: +39-06-4441197 or +39-06-4441198
Windrose Hotel (3 stars), Via Gaeta 39, tel: +39-06-4821913
Hotel Fiume (3 stars), Via Brescia 5, tel: +39-06-8543000
Hotel Sunrise (3 stars), Via Cilento 3, tel: +39-06-82011093
Hotel Virginia (2 stars), Via Montebello 94, tel: +39-06-4457689
Hotel Mirage (2 stars) , Via Milazzo 6, tel: +39-06-4455661 or +39-06-4463124
Hotel Marco Polo (2 stars), Via Magenta 39, tel: +39-06-44704478 or +39-06-4474091
Solomon Hotels (2 stars), Via Palestro 9, tel: +39-06-4465890, +39-06-44703927 or +39-06-484940
Hotel dell'Urbe (2 stars), Via dei Mille 27/a, tel: +39-06-4455767
Hotels in the historic centre of Rome
Grand Hotel Plaza (5 stars) Via del Corso 126, tel +39-06-69921111, +39-06-69941575
Hotel Colonna Palace (4 stars) P. Monte Citorio 12, tel. +39-06-675191
Hotel Piranesi (4 stars) Via del Babbuino 196, tel +39-06-328041
Hotel Carriage (3 stars) Via delle Carrozze 36, tel. +39-06-6990124
Hotel del Corso (3 stars) Via del Corso, 79, tel. +39-06-36006233, 06-36006041
Hotel Madrid (3 stars) Via M. de Fiori 93-95, tel +39-06-6991510
Hotels in the historic centre of Rome
Grand Hotel Plaza (5 stars) Via del Corso 126, tel +39-06-69921111, +39-06-69941575
Hotel Colonna Palace (4 stars) P. Monte Citorio 12, tel. +39-06-675191
Hotel Piranesi (4 stars) Via del Babbuino 196, tel +39-06-328041
Hotel Carriage (3 stars) Via delle Carrozze 36, tel. +39-06-6990124
Hotel del Corso (3 stars) Via del Corso, 79, tel. +39-06-36006233, 06-36006041
Hotel Madrid (3 stars) Via M. de Fiori 93-95, tel +39-06-6991510