Instrumental conditioning driven by neutral stimuli: A model tested with a simulated robotic rat

Current models of reinforcement learning are based on the assumption that learning must be guided by rewarding (unconditioned) stimuli. On the other hand, there is empirical evidence that dopamine bursts, which are commonly considered as the reinforcement learning signals, can also be triggered by apparently neutral stimuli, and that this can lead to conditioning phenomena in absence of any rewarding stimuli. In this paper we present a computational model, based on an hypothesis proposed in Redgrave and Gurney (2006), in which dopamine release is directly triggered by the superior colliculus (a dorsal midbrain structure) when it detects novel visual stimuli and this supports instrumental conditioning similarly to that usually ascribed to rewarding stimuli. The model incorporates various biological constraints, for example the anatomical and physiological data related to the micro-architecture of the superior colliculus presented in Binns and Salt (1997). The model is validated by reproducing with a simulated robotic rat the results of an experiment with real rats on the role of intrinsically reinforcing properties of apparently neutral stimuli reported in Reed et al. (1996).