Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Phil. Trans. R. Soc. B 362, 933–942 (2007).
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
Blanchard, T. C. & Gershman, S. J. Pure correlates of exploration and exploitation in the human brain. Cogn. Affect. Behav. Neurosci. 18, 117–126 (2018).
Hogeveen, J. et al. The neurocomputational bases of explore-exploit decision-making. Neuron 110, 1869–1879 (2022).
Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73, 595–607 (2012).
Gershman, S. J. Uncertainty and exploration. Decision 6, 277–286 (2019).
Trudel, N. et al. Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex. Nat. Hum. Behav. 5, 83–98 (2021).
Tomov, M. S., Truong, V. Q., Hundia, R. A. & Gershman, S. J. Dissociable neural correlates of uncertainty underlie different exploration strategies. Nat. Commun. 11, 2371 (2020).
Zajkowski, W. K., Kossut, M. & Wilson, R. C. A causal role for right frontopolar cortex in directed, but not random, exploration. eLife 6, e27430 (2017).
Pearson, J. M., Hayden, B. Y., Raghavachari, S. & Platt, M. L. Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr. Biol. 19, 1532–1537 (2009).
Barack, D. L., Chang, S. W. C. & Platt, M. L. Posterior cingulate neurons dynamically signal decisions to disengage during foraging. Neuron 96, 339–347 (2017).
Costa, V. D., Mitz, A. R. & Averbeck, B. B. Subcortical substrates of explore-exploit decisions in primates. Neuron 103, 533–545 (2019).
Alexandra Kredlow, M., Fenster, R. J., Laurent, E. S., Ressler, K. J. & Phelps, E. A. Prefrontal cortex, amygdala, and threat processing: implications for PTSD. Neuropsychopharmacology 47, 247–259 (2022).
Pitman, R. K. et al. Biological studies of post-traumatic stress disorder. Nat. Rev. Neurosci. 13, 769–787 (2012).
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L. & Platt, M. L. A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42, 1931–1939 (2017).
Pulcu, E. & Browning, M. The misestimation of uncertainty in affective disorders. Trends Cogn. Sci. 23, 865–875 (2019).
Levy, I. & Schiller, D. Neural computations of threat. Trends Cogn. Sci. 25, 151–171 (2021).
Ebitz, R. B., Albarran, E. & Moore, T. Exploration disrupts choice-predictive signals and alters dynamics in prefrontal cortex. Neuron 97, 475 (2018).
Costa, V. D. & Averbeck, B. B. Primate orbitofrontal cortex codes information relevant for managing explore–exploit tradeoffs. J. Neurosci. 40, 2553–2561 (2020).
Ogasawara, T. et al. A primate temporal cortex-zona incerta pathway for novelty seeking. Nat. Neurosci. 25, 50–60 (2022).
Aquino, T. G., Cockburn, J., Mamelak, A. N., Rutishauser, U. & O’Doherty, J. P. Neurons in human pre-supplementary motor area encode key computations for value-based choice. Nat. Hum. Behav. 7, 970–985 (2023).
Domenech, P., Rheims, S. & Koechlin, E. Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex. Science 369, eabb0184 (2020).
Aquino, T. G., Courellis, H., Mamelak, A. N., Rutishauser, U. & O′Doherty, J. P. Encoding of predictive associations in human prefrontal and medial temporal neurons during pavlovian appetitive conditioning. J. Neurosci. 44, e1628232024 (2024).
Barberini, C. L., Morrison, S. E., Saez, A., Lau, B. & Salzman, C. D. Complexity and competition in appetitive and aversive neural circuits. Front. Neurosci. 6, 170 (2012).
Herry, C. & Johansen, J. P. Encoding of fear learning and memory in distributed neuronal circuits. Nat. Neurosci. 17, 1644–1654 (2014).
Janak, P. H. & Tye, K. M. From circuits to behaviour in the amygdala. Nature 517, 284–292 (2015).
Pape, H. C. & Pare, D. Plastic synaptic networks of the amygdala for the acquisition, expression, and extinction of conditioned fear. Physiol. Rev. 90, 419–463 (2010).
Blanco, N. J., Otto, A. R., Maddox, W. T., Beevers, C. G. & Love, B. C. The influence of depression symptoms on exploratory decision-making. Cognition 129, 563–568 (2013).
de Berker, A. O. et al. Computations of uncertainty mediate acute stress responses in humans. Nat. Commun. 7, 10996 (2016).
Browning, M., Behrens, T. E., Jocham, G., O’Reilly, J. X. & Bishop, S. J. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat. Neurosci. 18, 590–596 (2015).
Aberg, K. C., Toren, I. & Paz, R. A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety. Mol. Psychiatry 27, 1573–1587 (2022).
Fan, H., Gershman, S. J. & Phelps, E. A. Trait somatic anxiety is associated with reduced directed exploration and underestimation of uncertainty. Nat. Hum. Behav. 7, 102–113 (2023).
Aberg, K. C. & Paz, R. The neurobehavioral correlates of exploration without learning: trading off value for explicit, prospective, and variable information gains. Cell Rep. 43, 113880 (2024).
Carleton, R. N. Into the unknown: a review and synthesis of contemporary models involving uncertainty. J. Anxiety Disord. 39, 30–43 (2016).
Ölveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005).
Mandelblat-Cerf, Y., Paz, R. & Vaadia, E. Trial-to-trial variability of single cells in motor cortices is dynamically modified during visuomotor adaptation. J. Neurosci. 29, 15053–15062 (2009).
Maimon, G. & Assad, J. A. Beyond Poisson: increased spike-time regularity across primate parietal cortex. Neuron 62, 426–440 (2009).
Stein, R. B., Gossen, E. R. & Jones, K. E. Neuronal variability: noise or part of the signal? Nat. Rev. Neurosci. 6, 389–397 (2005).
Faisal, A. A., Selen, L. P. & Wolpert, D. M. Noise in the nervous system. Nat. Rev. Neurosci. 9, 292–303 (2008).
Echeveste, R. & Lengyel, M. The redemption of noise: inference with neural populations. Trends Neurosci. 41, 767–770 (2018).
Krueger, P. M., Wilson, R. C. & Cohen, J. D. Strategies for exploration in the domain of losses. Judgm. Decis. Making 12, 104–117 (2017).
Lejarraga, T. & Hertwig, R. How the threat of losses makes people explore more than the promise of gains. Psychon. Bull. Rev. 24, 708–720 (2017).
Seymour, B., Maruyama, M. & De Martino, B. When is a loss a loss? Excitatory and inhibitory processes in loss-related decision-making. Curr. Opin. Behav. Sci. 5, 122–127 (2015).
Amaya, K. A. & Smith, K. S. Neurobiology of habit formation. Curr. Opin. Behav. Sci. 20, 145–152 (2018).
Cockburn, J., Man, V., Cunningham, W. A. & O’Doherty, J. P. Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain. Neuron 110, 2691–2702 (2022).
Unruh-Pinheiro, A. et al. Single-neuron correlates of decision confidence in the human medial temporal lobe. Curr. Biol. 30, 4722–4732 (2020).
Mormann, F., Bausch, M., Knieling, S. & Fried, I. Neurons in the human left amygdala automatically encode subjective value irrespective of task. Cereb. Cortex 29, 265–272 (2019).
Aquino, T. G. et al. Value-related neuronal responses in the human amygdala during observational learning. J. Neurosci. 40, 4761–4772 (2020).
Kehl, M. S. et al. Single-neuron representations of odours in the human brain. Nature 634, 626–634 (2024).
Reitich-Stolero, T. et al. Aversive generalization in human amygdala neurons. Curr. Biol. 35, 1137–1144 (2025).
Madarasz, T. J. et al. Evaluation of ambiguous associations in the amygdala by learning the structure of the environment. Nat. Neurosci. 19, 965–972 (2016).
Herry, C. et al. Processing of temporal unpredictability in human and animal amygdala. J. Neurosci. 27, 5958–5966 (2007).
Livneh, U. & Paz, R. Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories. Neuron 75, 133–142 (2012).
Monosov, I. E. How outcome uncertainty mediates attention, learning, and decision-making. Trends Neurosci. 43, 795–809 (2020).
Jezzini, A., Bromberg-Martin, E. S., Trambaiolli, L. R., Haber, S. N. & Monosov, I. E. A prefrontal network integrates preferences for advance information about uncertain rewards and punishments. Neuron 109, 2339–2352 (2021).
Kolling, N. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat. Neurosci. 19, 1280–1285 (2016).
Kobayashi, K. & Kable, J. W. Neural mechanisms of information seeking. Neuron 112, 1741–1756 (2024).
Litwin-Kumar, A. & Doiron, B. Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 1498–1505 (2012).
Shadlen, M. N. & Newsome, W. T. The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J. Neurosci. 18, 3870–3896 (1998).
Rubin, R., Abbott, L. F. & Sompolinsky, H. Balanced excitation and inhibition are required for high-capacity, noise-robust neuronal selectivity. Proc. Natl Acad. Sci. USA 114, E9366–E9375 (2017).
Levy, D. R. et al. Dynamics of social representation in the mouse prefrontal cortex. Nat. Neurosci. 22, 2013–2022 (2019).
Aston-Jones, G. & Cohen, J. D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Ann. Rev. Neurosci. 28, 403–450 (2005).
Payzan-LeNestour, E., Dunne, S., Bossaerts, P. & O’Doherty, J. P. The neural representation of unexpected uncertainty during value-based decision making. Neuron 79, 191–201 (2013).
Feng, Y. Y., Bromberg-Martin, E. S. & Monosov, I. E. Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making. Cell Rep. 43, 114341 (2024).
Grossman, C. D., Bari, B. A. & Cohen, J. Y. Serotonin neurons modulate learning rate through uncertainty. Curr. Biol. 32, 586–599 (2022).
Lottem, E. et al. Activation of serotonin neurons promotes active persistence in a probabilistic foraging task. Nat. Commun. 9, 1000 (2018).
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A. & Cohen, J. D. Humans use directed and random exploration to solve the explore-exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081 (2014).
Aberg, K. C. & Paz, R. Stress-induced avoidance in mood disorders. Nat. Hum. Behav. 6, 915–918 (2022).
Kahneman, D. & Tversky, A. Prospect theory: an analysis of decision under risk. Econometrica 47, 263 (1979).
Misra, A. et al. Methods for implantation of micro-wire bundles and optimization of single/multi-unit recordings from human mesial temporal lobe. J. Neural Eng. 11, 026013 (2014).
Fried, I. et al. Cerebral microdialysis combined with single-neuron and electroencephalographic recording in neurosurgical patients. J. Neurosurg. 91, 697–705 (1999).
Amunts, K., Mohlberg, H., Bludau, S. & Zilles, K. Julich-Brain: a 3D probabilistic atlas of the human brain’s cytoarchitecture. Science 369, 988–992 (2020).
Rutishauser, U. et al. Representation of retrieval confidence by single neurons in the human medial temporal lobe. Nat. Neurosci. 18, 1041–1050 (2015).
Kamiński, J., Brzezicka, A., Mamelak, A. N. & Rutishauser, U. Combined phase-rate coding by persistently active neurons as a mechanism for maintaining multiple items in working memory in humans. Neuron 106, 256–264 (2020).
Joshua, M., Elias, S., Levine, O. & Bergman, H. Quantifying the isolation quality of extracellularly recorded action potentials. J. Neurosci. Methods 163, 267–282 (2007).
Reitich-Stolero, T. & Paz, R. Affective memory rehearsal with temporal sequences in amygdala neurons. Nat. Neurosci. 22, 2050–2059 (2019).
Gershman, S. J. Deconstructing the human algorithms for exploration. Cognition 173, 34–42 (2018).
Frank, M. J., Doll, B. B., Oas-Terpstra, J. & Moreno, F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 12, 1062–1068 (2009).
Piray, P., Dezfouli, A., Heskes, T., Frank, M. J. & Daw, N. D. Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies. PLoS Comput. Biol. 15, e1007043 (2019).
Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies—revisited. Neuroimage 84, 971–985 (2014).
Wilson, R. C. & Collins, A. G. Ten simple rules for the computational modeling of behavioral data. eLife 8, e49547 (2019).
Shadlen, M. N., Hanks, T. D., Churchland, A. K., Kiani, R. & Yang, T. in Bayesian Brain: Probabilistic Approaches to Neural Coding (eds Doya, K. et al.) 209–237 (MIT Press, 2006).
Ison, M. J. et al. Selectivity of pyramidal cells and interneurons in the human medial temporal lobe. J. Neurophysiol. 106, 1713–1721 (2011).
Source link
