LOCEN Research Topic: Cognitive psychology experiments on intrinsic motivations

Research topic

Psychology defines IMs by contrasting them to extrinsic motivations. Extrinsic motivations (EMs) motivate behaviour (i.e., energise, focus, and guide its learning) to accomplish biologically relevant states/resources in the environment (e.g., achievement of food/water, pain avoidance) relevant for the organism's homoeostatic regulation and procreation (Baldassarre, 2011). In the case of robots, EMs can be related to the accomplishment of the user tasks assigned to the robot. In contrast, IMs drive behaviour and actions for their own sake, e.g. to gain knowledge about the world (curiosity, exploration), to be exposed to novel and surprising stimuli (Berlyne, 1960), to reach higher levels of competence in changing the environment with an agent's own actions (White, 1959; Ryan and Deci, 2000). In general, IMs can drive the acquisition of knowledge and skills in the absence of tasks directly established by biological fitness or by the robot users (e.g., to learn to master the manipulation of different objects without being instructed to do so). Later, however, such knowledge and skills can become very useful to increase fitness or solve user tasks, and this justifies the evolutionary emergence of IMs in organisms and their utility for robots (Singh et al., 2010; Baldassarre, 2011; Baldassarre and Mirolli, 2013).  Computational investigations are clarifying the specific mechanisms that might underlie IMs (both in robots and organisms; Schmidhuber et al., 1991, 2010; Barto et al., 2004, 2010, 2013; Oudeyer et al., 2007; Baldassarre et al., 2013). Here we adopt a threefold classification of such IM mechanisms (Oudeyer and Kaplan, 2007; Barto et al., 2013; Baldassarre and Mirolli, 2013): (a) novelty-based IMs: related to novel, previously non-experienced, stimuli; (b) prediction-based IMs: related to the violation of the agent's predictions; (c) competence-based IMs: related to action, i.e. to the agent's competence to change the world and accomplish its own goals. Importantly, note how IM mechanisms have in common the fact that they are related to an increase of the agent's knowledge (broadly defined). These mechanisms can serve different proximal functions within the robot controller/organism brain, e.g. to drive attention, produce learning signals, select actions that maximise learning progress, or generate goals autonomously (Baldassarre and Mirolli, 2013).

Research specific problems

• What are the specific psychological processes/mechanisms behind the different types of intrinsic motivations (surprise-based, novelty-based, competence-based)?
• What is the interplay of these processes, and how do they contribute, in an integrated fashion, to the cumulative learning of knowledge and skills during the whole life of people?  

Research method

• Psychological/cognitive experiments with primates, especially humans (human adults, children, apes/monkeys).

Examples of research of this type carried out by the group

(see full references below; the pdf files of the paper are retrievable from here)

• Caligiore et al. (in press).
• Polizzi di Sorrentino et al. (2014).
• Taffoni et al. (2014).

Requested motivations of the candidate

• Strong interest in the topic and motivation to carry out research on it (very important)
• Desire to acquire the knowledge and methods of the group

Requested knowledge of the candidate

Requested skills of the candidate

• Capacity to read and understand scientific papers in English
• (Possibly) capacity to contribute to the design of empirical experiments
• Capacity to recruit experimental participants and carry out cognitive psychology experiments
• Capacity to contribute to write reports in English

References

• Baldassarre, G. (2011). What are intrinsic motivations? A biological perspective. In IEEE ICDL 2011, e1-8.
• Baldassarre, G., Mannella, F., Fiore, V.G., Redgrave, P., Gurney, K., Mirolli, M. (2013). Intrinsically motivated action-outcome learning and goal-based action recall: A bio-constrained computational model. Neural Networks, 41, 168-187.
• Baldassarre, G., Mirolli, M. (eds.)(2013). Intrinsically motivated learning in natural and artificial systems. Berlin: Springer.Barto A., Mirolli M., Baldassarre G. (2013). Novelty or surprise? Frontiers in Cognitive Sci., 4 (907).
• Barto A.G., Singh S., Chentanez N. (2004), Intrinsically Motivated Learning of Hierarchical Collections of Skills International Conference on Developmental Learning (ICDL), LaJolla, CA, USA.Berlyne, D.E. (1960). Conflict, Arousal. and Curiosity. New York: McGraw-Hill. 
• Caligiore, D.; Mustile, M.; Cipriani, D.; Redgrave, P.; Triesch, J.; De Marsico, M. & Baldassarre, G. (in press). Intrinsic motivations driving learning of eye movements: an experiment with human adults. PLOS One. 
• Oudeyer P.-Y., Kaplan F. (2007). What is Intrinsic Motivation? A Typology of Computational Approaches. Frontiers in Neurorobotics, 1, 6. 
• Oudeyer P.-Y., Kaplan F., Hafner V.V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11 (2), 265-286.
• Polizzi di Sorrentino, E.; Sabbatini, G.; Truppa, V.; Bordonali, A.; Taffoni, F.; Formica, D.; Baldassarre, G.; Mirolli, M. & Guglielmelli, Eugenio, V. E. (2014). Exploration and learning in capuchin monkeys (Sapajus spp.): the role of actionoutcome contingencies. Animal Cognition, 17 (5), 1081-1088.
• Ryan R.M., Deci E.L. (2000). Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemp Educ Psychol, 25 (1), 54-67.
• Schmidhuber, J. (1991). Curious model-building control systems. International Joint Conference on Artificial Neural Networks, 2, 1458-1463.Schmidhuber, J. (2010). Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990-2010). IEEE TAMD, 2 (3), 230 -247.
• Singh S., Lewis R.L., Barto A.G., Sorg J. (2010). Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2, 70-82.
• Taffoni, F.; Tamilia, E.; Focaroli, V.; Formica, D.; Ricci, L.; Di Pino, G.; Baldassarre, G.; Mirolli, M.; Guglielmelli, E. & Keller, F. (2014). Development of goal-directed action selection guided by intrinsic motivations: an experiment with children. Experimental Brain Research, 232 (7), 2167-2177.
• White R.W. (1959). Motivation reconsidered: the concept of competence. Psych Review, 66, 297-333.