Home
 /
Papers
 /
Neuroscience
 /
A hardwired neural circuit for temporal difference learning
Preprint
Open access
Top 1% · Validated by QED

A hardwired neural circuit for temporal difference learning

Malcolm G. Campbell, Yongsoo Ra, Zhiqin Chen +5 more
October 2025
 ·
bioRxiv
 ·
DOI 
10.1101/2025.09.18.677203
99
%
QED Score
QED Impressions

By demonstrating that a hardwired biphasic D1-MSN–dopamine filter computes temporal difference errors—and that its parameters directly set the temporal discount factor—this paper transforms a decades-old algorithmic theory of learning into concrete circuit biology.

Run full QED analysis
Read paper
Save

QED claims analysis

Summary

The dopamine system implements temporal difference learning through a hardwired neural circuit in which phasic dopamine release selectively potentiates D1-MSN responses to reward-predicting cues, and D1-MSNs in turn drive dopamine neuron activity through a biphasic linear filter that computes temporal differences, with the balance of excitatory and inhibitory components setting the temporal discount factor that governs animals' preference for immediate versus delayed rewards.

See full analysis
Main Claim 1 –

NAc Dopamine-Axon Opto-Conditioning Generates TD Error-Like Dopamine Activity

Supporting evidence

  • Opto-Conditioning Potentiates D1-MSN Cue Responses Into Value Signals
  • Opto-Conditioning Induces Behavioral Discrimination Consistent With Learning
Main Claim 2 –

D1-MSNs Drive Dopamine via Biphasic Filter TD Transformation

Supporting evidence

  • lNAc Provides Value Signals to VTA Dopamine During Natural Learning
Main Claim 3 –

Temporal Difference Computation Is Ubiquitous Across Striatal Dopamine Circuits

Abstract

The neurotransmitter dopamine plays a major role in learning by acting as a teaching signal to update the brain’s predictions about rewards. A leading theory proposes that this process is analogous to a reinforcement learning algorithm called temporal difference (TD) learning, and that dopamine acts as the error term within the TD algorithm (TD error). Although many studies have demonstrated similarities between dopamine activity and TD errors, the mechanistic basis for dopaminergic TD learning remains unknown. Here, we combined large-scale neural recordings with patterned optogenetic stimulation to examine whether and how the key steps in TD learning are accomplished by the circuitry connecting dopamine neurons and their targets. Replacing natural rewards with optogenetic stimulation of dopamine axons in the nucleus accumbens (NAc) in a classical conditioning task gradually generated TD error-like activity patterns in dopamine neurons by specifically modifying the task-related activity of NAc neurons expressing the D1 dopamine receptor (D1 neurons). In turn, patterned optogenetic stimulation of NAc D1 neurons in naïve animals drove dopamine neuron spiking according to the TD error of the stimulation pattern, indicating that TD computations are hardwired into this circuit. The transformation from D1 neurons to dopamine neurons could be described by a biphasic linear filter, with a rapid positive and delayed negative phase, that effectively computes a temporal difference. This finding suggests that the time horizon over which the TD algorithm operates—the temporal discount factor—is set by the balance of the positive and negative components of the linear filter, pointing to a circuit-level mechanism for temporal discounting. These results provide a new conceptual framework for understanding how the computations and parameters governing animal learning arise from neurobiological components.

Related papers

How the QED Score works

QED Score is an AI-based quality metric for life science research. It rates each paper on two dimensions: originality (is the finding new and important?) and validity (does the evidence support the claims?). Author names and affiliations are removed before scoring. Each manuscript is broken down into its claims and the experiments behind them, and specialized AI agents then examine those claims for statistical rigor, consistency across figures, and contradictions with the existing literature. A verification layer re-checks every issue they flag. Scores are percentile ranks against all papers in the corpus. In a blinded study of papers where QED Score and journal rank disagreed, domain experts sided with QED Score in 75% of cases.
Read the methodology

Free access for academic researchers

Create your free QED account to validate your research, strengthen grant proposals, and uncover scientific insights.
We've sent you an access link.
Please check your inbox.

Didn't get your email? Check your spam folder or reach out to info@qedscience.com

Oops! Something went wrong while submitting the form.
Looking for QED for pharma, biotech or life science organizations?