University Biology — Year 3 · Bachelor Year 3

19Neural Plasticity, Learning and Memory

In 1953 a surgeon removed the inner part of both temporal lobes of a young man with intractable epilepsy. The seizures abated; the patient, known for fifty years as H.M., never formed another lasting memory of an event. He could hold a conversation, learn to trace a star in a mirror as well as anyone — and each day deny ever having seen the task — and he remembered his childhood. Memory, it turned out, was not one thing, and the making of new memories of events depended on a structure the size of a thumb. What changes in a brain when it learns is the oldest question of neuroscience, and its answer, worked out in a sea slug with twenty thousand neurons and in slices of rat hippocampus, is that synapses change their strength according to what passes through them — a rule Donald Hebb guessed in 1949 and a molecule, the NMDA receptor, turned out to implement. This chapter treats that rule and its machinery, the circuits that use it to store places and events, the mathematics that says what such a system can learn and how much it can hold, and the reward signal that tells it what is worth learning.

19.1 Kinds of memory and where they live

Definition 19.1 (Forms of learning and memory)

The simplest learning is non-associative: habituation, a declining response to a repeated harmless stimulus, and sensitisation, an enhanced response after a harmful one. Associative learning links two events: in classical conditioning (Pavlov) a neutral stimulus that predicts a reward or punishment comes to elicit the response; in operant conditioning an action that is rewarded is repeated. Human memory divides into declarative memory — facts and events, consciously recalled, dependent on the hippocampus and the medial temporal lobe — and non-declarative memory — skills and habits (striatum, cerebellum), conditioned fear (amygdala), priming (cortex) — expressed in performance without recollection. A memory is held first in a labile short-term form, lasting seconds to hours, and is consolidated over hours to years into a stable long-term form that no longer needs the hippocampus and resides in the cortex.

Evidence. Scoville and Milner (1957) reported H.M.: after removal of both hippocampi and the neighbouring cortex, his intelligence and his memories from before the operation were largely intact, his working memory normal for as long as he attended to something, but he could form no new memory of any event or fact. He learned motor skills at a normal rate over days while denying having practised — so skill memory did not need the hippocampus — and he remembered the remote past better than the years just before surgery, so the hippocampus was needed to lay down and for a time to hold declarative memories, not to store them for ever. Monkeys and rats with matching lesions showed the same dissociation, and the study of a single patient reorganised the field. ∎

19.2 Hebb’s rule and its molecule

Definition 19.2 (Hebbian plasticity, LTP and LTD)

Hebb (1949) proposed that when a presynaptic neuron repeatedly takes part in firing a postsynaptic one, the connection between them is strengthened — “cells that fire together wire together”. Long-term potentiation (LTP) is its experimental form: a brief burst of high-frequency stimulation of a pathway leaves the synapses it activated stronger for hours in a slice and for weeks in an animal. LTP is input-specific (only the stimulated synapses change), associative (a weak input paired with a strong one to the same cell is potentiated) and needs cooperativity among inputs to depolarise the cell. Long-term depression (LTD) is its opposite, produced by prolonged low-frequency activity. In spike-timing-dependent plasticity the sign is set by order: a presynaptic spike a few milliseconds before the postsynaptic one strengthens the synapse, one just after weakens it, within a window of about $20\,\mathrm{ms}$ on either side — so that a synapse learns to predict, not merely to correlate.

Proposition 19.3 (The NMDA receptor as coincidence detector)

At excitatory synapses glutamate acts on two receptors. The AMPA receptor opens on binding and carries the ordinary synaptic current. The NMDA receptor binds glutamate but is blocked at resting potential by a magnesium ion in its pore; only when the postsynaptic membrane is depolarised — by many inputs at once, or by a back-propagating action potential — is the magnesium expelled and the channel conducts, letting in calcium. The NMDA receptor thus opens only when presynaptic release (glutamate) coincides with postsynaptic activity (depolarisation): it is Hebb’s “and” gate in a single protein. The calcium that enters a dendritic spine activates the kinase CaMKII, which phosphorylates AMPA receptors and drives more of them into the synapse from a reserve — the synapse is stronger within minutes (early LTP). A large, slow calcium rise instead activates the phosphatase calcineurin, which removes AMPA receptors: LTD. Potentiation lasting more than a few hours (late LTP) needs new protein: kinases reach the nucleus, the transcription factor CREB switches on genes, and the spine grows and may split — the same requirement for gene expression that separates short-term from long-term memory in every animal studied.

Evidence. Bliss and Lømo (1973) stimulated the perforant path into the rabbit hippocampus with a train of pulses and found the evoked response enlarged for hours afterwards — the first LTP. Collingridge (1983) showed that a blocker of the NMDA receptor (AP5) prevented the induction of LTP but not the ordinary synaptic response; Morris (1986) infused AP5 into the hippocampus of rats and found that they could no longer learn the position of a hidden platform in a pool of water while swimming and seeing normally; and mice in which the NMDA receptor was deleted only in the CA1 region of the hippocampus lacked both LTP there and spatial memory. The same receptor, in the same place, was necessary for the synaptic change and the learning. ∎

The NMDA receptor as Hebb’s gate. Glutamate alone opens AMPA receptors; the NMDA channel conducts only when the spine is also depolarised and its magnesium leaves. The calcium that then enters sets the synapse’s future: a fast large rise strengthens it, a slow small one weakens it.

Left: long-term potentiation — after a brief tetanus the stimulated pathway’s response stays enlarged for hours while an unstimulated pathway to the same cells is unchanged (input specificity). Right: the spike-timing window — the synapse strengthens if the presynaptic spike leads and weakens if it lags.

Theorem 19.4 (What a Hebbian synapse learns)

Let a neuron receive inputs $\mathbf{x} = (x_{1},\dots,x_{n})$ through weights $\mathbf{w}$ and respond linearly, $y = \mathbf{w}\cdot\mathbf{x}$ . Hebb’s rule, $\Delta\mathbf{w} = \eta\, y\,\mathbf{x}$ , gives on average over the inputs

\langle\Delta\mathbf{w}\rangle = \eta\,C\,\mathbf{w}, \qquad C = \langle\mathbf{x}\mathbf{x}^{\mathsf{T}}\rangle ,

where $C$ is the correlation matrix of the inputs. The weights grow without bound, and grow fastest along the eigenvector of $C$ with the largest eigenvalue: the synapse comes to detect the pattern of input that is most often present — the principal component of its inputs. With a term that penalises large weights, Oja’s rule $\Delta\mathbf{w} = \eta\, y\,(\mathbf{x} - y\,\mathbf{w})$ , the weight vector converges to that eigenvector normalised to unit length, and the neuron becomes a stable detector of the dominant correlation in what it sees.

Proof. Substituting $y = \mathbf{w}\cdot\mathbf{x} = \mathbf{x}^{\mathsf{T}} \mathbf{w}$ into Hebb’s rule gives $\Delta\mathbf{w} = \eta\,\mathbf{x} \mathbf{x}^{\mathsf{T}}\mathbf{w}$ , whose average over inputs (with $\mathbf{w}$ changing slowly) is $\eta C\mathbf{w}$ . $C$ is symmetric and positive semi-definite, so it has orthogonal eigenvectors $\mathbf{e}_{k}$ with eigenvalues $\lambda_{k} \ge 0$ ; writing $\mathbf{w} = \sum_{k} a_{k}\mathbf{e}_{k}$ , each component obeys $\Delta a_{k} = \eta\lambda_{k} a_{k}$ and grows as $(1 + \eta\lambda_{k})^{t}$ , so the component along the largest eigenvalue comes to dominate the direction of $\mathbf{w}$ while its length diverges. Oja’s rule averages to $\eta(C\mathbf{w} - (\mathbf{w}^{\mathsf{T}}C\mathbf{w})\mathbf{w})$ ; at a fixed point $C\mathbf{w} = (\mathbf{w}^{\mathsf{T}}C\mathbf{w})\mathbf{w}$ , so $\mathbf{w}$ is an eigenvector with $\lambda = \mathbf{w}^{\mathsf{T}} C\mathbf{w} = \lambda|\mathbf{w}|^{2}$ , hence $|\mathbf{w}| = 1$ ; the fixed point along the largest eigenvalue is the stable one (a perturbation toward another eigenvector shrinks, since that eigenvector’s eigenvalue is smaller), which is admitted here. ∎

Example 19.5 (Two inputs)

A neuron receives two inputs that are usually active together, so $C = \begin{pmatrix} 1 & 0.8 \\ 0.8 & 1\end{pmatrix}$ . The eigenvectors are $(1,1)/\sqrt{2}$ with eigenvalue $1.8$ and $(1,-1)/\sqrt{2}$ with eigenvalue $0.2$ . Starting from $\mathbf{w} = (1, 0)$ , Hebb’s rule after many small steps turns $\mathbf{w}$ toward $(1,1)$ : the neuron learns to respond to the two inputs together, and to ignore the rare occasions on which only one fires — it has extracted the correlation. With Oja’s rule the weights settle at $(0.71, 0.71)$ . This is the sense in which Hebbian synapses learn what goes together, and it is why the associative property of LTP, pairing a weak input with a strong one, follows from the rule.

19.3 A slug that learns

Proposition 19.6 (Sensitisation in Aplysia)

The sea slug Aplysia withdraws its gill when its siphon is touched, a reflex of a few dozen identifiable neurons. A shock to the tail sensitises it: the withdrawal to a light touch is enhanced, for minutes after one shock and for weeks after several. Kandel and colleagues (1970s–1990s) traced the change to the synapse between the siphon’s sensory neuron and the gill’s motor neuron. The tail shock excites an interneuron that releases serotonin onto the sensory neuron’s terminal; serotonin raises cyclic AMP, which activates protein kinase A, which phosphorylates and closes a potassium channel; the terminal’s action potentials broaden, more calcium enters, and more transmitter is released per spike — presynaptic facilitation, a covalent change lasting minutes. Repeated shocks send the kinase to the nucleus, where it activates CREB, which switches on genes that build new synaptic terminals: the long-term memory is more synapses, and it is blocked by inhibitors of protein synthesis given during training but not afterwards. Habituation is the opposite, a depression of the same synapse; and classical conditioning, pairing the touch with the shock, enhances facilitation at the synapses that were active just before the serotonin arrived — a Hebbian coincidence implemented presynaptically, by an adenylyl cyclase that is stimulated by calcium and serotonin together.

The sensitisation circuit of Aplysia. Serotonin from an interneuron acts on the sensory terminal, and the same synapse carries the short-term memory as a phosphorylated channel and the long-term memory as new terminals. — The sensitisation circuit of *Aplysia*. Serotonin from an interneuron acts on the sensory terminal, and the same synapse carries the short-term memory as a phosphorylated channel and the long-term memory as new terminals.

Left: Aplysia californica, whose few and large neurons let the synapse of a memory be found and recorded. Right: a dendrite studded with spines, each the postsynaptic side of one excitatory synapse and each a compartment whose calcium decides its own fate. — Left: *Aplysia californica*, whose few and large neurons let the synapse of a memory be found and recorded. Right: a dendrite studded with spines, each the postsynaptic side of one excitatory synapse and each a compartment whose calcium decides its own fate.

19.4 Circuits for places and events

Definition 19.7 (The hippocampal circuit)

Cortical input reaches the hippocampus through the entorhinal cortex and passes through three stages: the dentate gyrus, whose granule cells outnumber their inputs and fire sparsely (separating similar patterns); CA3, whose pyramidal cells connect recurrently to one another, each receiving thousands of synapses from others — an autoassociative network that can complete a stored pattern from a fragment; and CA1, which compares the CA3 output with the direct input and returns the result to the cortex. In a rat exploring an arena, a hippocampal pyramidal cell fires only when the animal is in one place — a place cell (O’Keefe, 1971) — and a few hundred of them map the arena; in the entorhinal cortex upstream, grid cells (Moser, 2005) fire at the vertices of a hexagonal lattice covering the space, a coordinate system from which place fields are built. During sleep and rest the sequences of place cells that fired during a run are replayed at high speed, and blocking the replay impairs the memory of the run: the hippocampus rehearses the day’s paths to the cortex.

Left: the hippocampal loop — separation in the dentate gyrus, storage and completion in the recurrent CA3, comparison in CA1. Right: each place cell fires in one region of an arena; together they map the space.

Proposition 19.8 (How much an autoassociative network holds)

A network of $N$ neurons, each connected to all the others with Hebbian weights $w_{ij} \propto \sum_{\mu}\xi_{i}^{\mu}\xi_{j}^{\mu}$ summed over the stored binary patterns $\xi^{\mu}$ , retrieves a stored pattern from a corrupted or partial cue by settling into it as an attractor (Hopfield, 1982). It does so reliably for up to about $P \approx 0.14N$ random patterns; beyond that the patterns interfere and retrieval collapses. A rat’s CA3, with some $3\times 10^{5}$ pyramidal cells, could on this estimate hold some $4\times 10^{4}$ patterns — the order of the distinct places or episodes an animal meets in a season — and its sparse firing (a few per cent of cells active in any pattern) raises the capacity further. Pattern completion from a cue is what lets a smell or a corner recall a whole scene; the price is that similar memories merge, which the dentate gyrus’s separation resists.

Proof. Admitted at this level. ∎

Method 19.9 (Testing memory and its mechanism)

(1) A spatial task: the Morris water maze — a rat swims in opaque water to a platform hidden at a fixed place, learns its position from the room’s cues over days, and is tested by the time spent searching the right quadrant when the platform is removed; hippocampal lesions, NMDA blockers and the loss of LTP each abolish the learning. (2) An associative task: fear conditioning — a tone paired with a foot shock comes to elicit freezing; the memory depends on the amygdala, and a context-specific version on the hippocampus. (3) A mechanism test: block the candidate (a drug, a knockout restricted to one region and time) and show that learning fails while perception and movement do not. (4) The engram: label the neurons active during learning with a light-sensitive channel (Chapter 17) and later reactivate them with light — the animal freezes in a safe context, as if remembering; silencing them blocks recall. Memory is in identifiable cells and their synapses, and can be switched on and off.

A section through the rodent hippocampus: the curved band of pyramidal cells (green) from CA3 to CA1 and the interlocking dentate gyrus (red). Every spatial memory of the animal passes through this loop.

19.5 Reward, prediction and the plasticity of a lifetime

Proposition 19.10 (Learning by prediction error)

Let $V$ be the value an animal attaches to a cue and $\lambda$ the reward that follows it. The Rescorla–Wagner rule (1972) says that on each trial the value changes by a fraction $\alpha$ of the prediction error:

\Delta V = \alpha\,(\lambda - V), \qquad V_{n} = \lambda\bigl(1 - (1-\alpha)^{n}\bigr) ,

so learning is fast at first and slows as the prediction improves, stops when the reward is fully predicted, and reverses (extinction) when the reward is withheld. When two cues are presented together they share one prediction, so a cue that adds nothing to a prediction already made is not learned (blocking). Schultz (1997) recorded dopamine neurons of the midbrain in monkeys and found that they fire to an unexpected reward, fall silent to a fully predicted one, and dip below baseline when a predicted reward fails to arrive: they broadcast $\lambda - V$ , the prediction error itself, to the striatum and cortex, where dopamine gates plasticity at the synapses that were recently active. Learning what predicts reward is Hebbian plasticity with a third factor that says whether the outcome was better or worse than expected — and drugs of abuse, by driving dopamine directly, counterfeit a permanent “better than expected”.

Proof. The recursion $V_{n+1} = V_{n} + \alpha(\lambda - V_{n})$ gives $\lambda - V_{n+1} = (1-\alpha)(\lambda - V_{n})$ , so the error shrinks geometrically, $\lambda - V_{n} = (1-\alpha)^{n}\lambda$ from $V_{0} = 0$ , whence the formula. For two cues $A$ and $B$ presented together the prediction is $V_{A} + V_{B}$ ; if $A$ was trained first to $V_{A} = \lambda$ , the error on compound trials is zero and $B$ acquires nothing. ∎

Left: the Rescorla–Wagner learning curve with = 0.2 — geometric approach to the reward’s value, then extinction when the reward stops. Right: dopamine neurons report the prediction error — a burst for a surprise, nothing for a predicted reward, a dip when a predicted reward is missed. — Left: the Rescorla–Wagner learning curve with $\alpha = 0.2$ — geometric approach to the reward’s value, then extinction when the reward stops. Right: dopamine neurons report the prediction error — a burst for a surprise, nothing for a predicted reward, a dip when a predicted reward is missed.

Definition 19.11 (Critical periods and the diseases of plasticity)

Plasticity is greatest in windows of development. Hubel and Wiesel (1960s) covered one eye of a kitten for a few weeks after birth: the visual cortex, normally driven by both eyes in alternating columns, became driven almost entirely by the open eye, and the deprived eye stayed functionally blind for life — while the same deprivation in an adult cat changed nothing. Such critical periods, for vision, for language, for the songs of birds, close when inhibitory circuits mature and the extracellular matrix stiffens, and they can be reopened a little by drugs and training. Plasticity’s pathologies are of dose and target: addiction is learning driven by a counterfeit prediction error, which builds habits that outlast the pleasure; post-traumatic stress is a fear memory over-consolidated by stress hormones; and Alzheimer’s disease begins where declarative memory begins, in the entorhinal cortex and hippocampus, with the loss of synapses — which tracks the dementia better than any plaque count — before the loss of neurons.

Remark 19.12 (What memory is made of)

A memory is not a recording but a change in the strengths and numbers of synapses, distributed over the cells that were active when it was made, laid down by a rule that strengthens what fires together, gated by a signal that says whether the outcome mattered, and rehearsed in sleep until the cortex holds it on its own. The rule is Hebb’s, the gate is the NMDA receptor, the signal is dopamine, the rehearsal is replay, and the mathematics says what such a system can hold and why it confuses the similar. Every experiment that has switched a memory on or off — by a blocker, a gene, a light — has found it in those synapses, in those cells. H.M. died in 2008; his brain, sectioned and imaged, showed the lesion that had taught the field, and no more.

19.6 Exercises

Exercise 19.1 ★

Distinguish declarative from non-declarative memory with H.M.’s performance, and name a brain region for each of four kinds of non-declarative memory.

Solution

Solution of Exercise 19.1.

Declarative memory (facts, events) was abolished in H.M. — he could recall nothing new — while non-declarative memory survived: he learned mirror drawing normally without remembering the sessions. Skills: striatum and cerebellum; conditioned fear: amygdala; conditioned motor reflexes: cerebellum; priming: sensory cortex.

Exercise 19.2 ★

State Hebb’s postulate and the three properties of LTP that follow from it.

Solution

Solution of Exercise 19.2.

When a presynaptic cell repeatedly contributes to firing a postsynaptic cell, their connection strengthens. Hence LTP is input-specific (only the active synapses, which took part, change), associative (a weak input active while the cell fires under a strong one is strengthened), and cooperative (enough inputs must act together to fire the cell).

Exercise 19.3 ★

Explain why the NMDA receptor is a coincidence detector, and what happens to LTP when its blocker AP5 is applied before, and after, the tetanus.

Solution

Solution of Exercise 19.3.

It conducts only when glutamate is bound (the presynaptic cell fired) and the membrane is depolarised enough to expel the magnesium (the postsynaptic cell is active): both conditions of Hebb’s rule in one protein. AP5 before the tetanus: no calcium entry, no LTP, though transmission through AMPA receptors is normal. AP5 after: LTP already induced is unaffected — the receptor is needed for induction, not maintenance.

Exercise 19.4 ★

In Aplysia, what distinguishes short-term from long-term sensitisation at the molecular level, and what experiment shows it?

Solution

Solution of Exercise 19.4.

Short-term: covalent modification of existing proteins — PKA phosphorylates and closes a potassium channel, broadening the spike and increasing release; no new protein. Long-term: PKA activates CREB, new genes are transcribed and new synaptic terminals grow. A protein-synthesis inhibitor given during repeated training leaves the short-term facilitation intact and abolishes the memory measured days later; given after, it does nothing.

Exercise 19.5 ★★

With $\alpha = 0.3$ , how many trials until $V$ reaches $90\,\%$ of $\lambda$ ? If the reward is then withheld, how many trials until $V$ falls below $10\,\%$ ? Why are acquisition and extinction symmetric in this model, and are they in animals?

Solution

Solution of Exercise 19.5.

$0.7^{n} \le 0.1$ : $n \ge 6.5$ , so $7$ trials. From $0.9\lambda$ , $0.9\times 0.7^{n} < 0.1$ : again $7$ trials. Symmetric because the same $\alpha$ governs both directions. In animals extinction is usually slower and the original memory returns after a rest (spontaneous recovery): extinction is new learning that suppresses the old, not its erasure.

Exercise 19.6 ★★

Two inputs have $C = \begin{pmatrix}1 & 0.5\\ 0.5 & 1\end{pmatrix}$ . Find the eigenvectors and eigenvalues, and say toward which direction Hebb’s rule turns the weights and what the neuron then detects. Apply Oja’s rule for one step from $\mathbf{w} = (1,0)$ with input $\mathbf{x} = (1,1)$ and $\eta = 0.1$ .

Solution

Solution of Exercise 19.6.

Eigenvectors $(1,1)/\sqrt{2}$ with eigenvalue $1.5$ and $(1,-1)/\sqrt{2}$ with $0.5$ ; Hebb turns $\mathbf{w}$ toward $(1,1)$ , and the neuron comes to detect the two inputs firing together. Oja’s step: $y = 1$ , $\Delta\mathbf{w} = 0.1\times 1\times((1,1) - 1\times(1,0)) = (0,0.1)$ , so $\mathbf{w} = (1, 0.1)$ .

Exercise 19.7 ★★

A spine has a volume of $0.1\,\mathrm{fL}$ . Entry of $500$ calcium ions raises its free concentration by how much (in $\text{µ}\mathrm{mol}/\mathrm{L}$ ), if $95\,\%$ are immediately bound by buffers? Compare with the resting $0.1\,\text{µ}\mathrm{mol}/\mathrm{L}$ and with the $K_{d}$ of CaMKII’s activator calmodulin, about $1\,\text{µ}\mathrm{mol}/\mathrm{L}$ .

Solution

Solution of Exercise 19.7.

$500\times 0.05 = 25$ free ions; $25/6\times 10^{23} = 4.2\times 10^{-23}$ mol in $10^{-16}$ L: $0.42\,\text{µ}\mathrm{mol}/\mathrm{L}$ , four times the resting level but below calmodulin’s $K_{d}$ — partial activation only; a few channel openings together are needed to reach the micromolar range that switches CaMKII on.

Exercise 19.8 ★★

Explain blocking with the Rescorla–Wagner rule and give the dopamine recording that corresponds to each stage.

Solution

Solution of Exercise 19.8.

A is trained until $V_{A} = \lambda$ : dopamine bursts on the early trials and falls silent as the reward becomes predicted. On compound trials the prediction $V_{A} + V_{B} = \lambda$ is already exact, the error is zero, dopamine is silent, and $V_{B}$ never changes: B is blocked. Tested alone, B evokes no response; if a reward then follows B alone it is unexpected and dopamine bursts.

Exercise 19.9 ★★

Predict the results of (a) NMDA receptor deletion restricted to CA1, (b) a protein-synthesis inhibitor given during training, (c) the same inhibitor given a day later, (d) optogenetic reactivation of the dentate-gyrus cells that were active during a fear conditioning, in a new context.

Solution

Solution of Exercise 19.9.

(a) No LTP in CA1 and a spatial-memory deficit, with other learning intact. (b) Short-term memory normal, long-term memory absent at $24\,\mathrm{h}$ . (c) No effect — consolidation is over (unless the memory is reactivated, when it becomes labile again). (d) The mouse freezes in the safe context: the memory is recalled artificially by reactivating the cells that encoded it.

Exercise 19.10 ★★★

Show that under Hebb’s rule alone the length of $\mathbf{w}$ grows without bound, and that Oja’s rule has $|\mathbf{w}| = 1$ at any fixed point. Why is unbounded growth a biological problem, and what mechanisms (synaptic scaling, LTD, sleep) might correspond to the normalising term?

Solution

Solution of Exercise 19.10.

$\Delta|\mathbf{w}|^{2} \approx 2\mathbf{w}\cdot\Delta\mathbf{w} = 2\eta y^{2} \ge 0$ : the length never decreases and grows whenever the neuron fires. At an Oja fixed point $C\mathbf{w} = (\mathbf{w}^{\mathsf{T}}C \mathbf{w})\mathbf{w}$ , so $\mathbf{w}$ is an eigenvector with $\lambda = \lambda|\mathbf{w}|^{2}$ , hence $|\mathbf{w}| = 1$ . Unbounded growth would saturate every synapse, abolish selectivity and drive runaway excitation; the biological normalisers are synaptic scaling (all of a neuron’s synapses scaled down when it is too active), LTD, and the down-scaling of synapses during sleep.

Exercise 19.11 ★★★

A rat’s CA3 has $3\times 10^{5}$ cells; a human’s about $2\times 10^{6}$ . Estimate the Hopfield capacity of each. Why is the estimate a poor measure of how many memories a person holds, and what does the hippocampus’s role in consolidation add to the picture?

Solution

Solution of Exercise 19.11.

Rat: $0.14\times 3\times 10^{5} \approx 4\times 10^{4}$ ; human: $0.14 \times 2\times 10^{6} \approx 3\times 10^{5}$ . The estimate assumes random dense patterns and full connectivity; real patterns are sparse (which raises capacity greatly) and structured, and the hippocampus is a temporary store that hands memories to a cortex of $10^{10}$ neurons — the lifetime store is the cortex, and the hippocampal number is a buffer size, not a memory count.

Exercise 19.12 ★★★

Kittens deprived of one eye’s vision for weeks lose that eye’s cortical territory; adults do not. Explain with Hebbian competition why the open eye wins, why binocular deprivation has a smaller effect than monocular, and what closes the critical period.

Solution

Solution of Exercise 19.12.

The open eye’s inputs are correlated with the cortical cell’s firing and are strengthened; the closed eye’s inputs are not, and under competition for a limited total weight they weaken — Hebb plus normalisation. With both eyes closed neither input is favoured, so both keep (weak) territory and the effect is milder. The period closes as inhibitory circuits mature, perineuronal nets stiffen around synapses, and the NMDA receptor’s subunits change to a less permissive form.

19.7 Problem: A Synapse That Remembers

Problem 19.1

Weekend problem — a memory followed from calcium in a spine to a rat in a maze: the calcium arithmetic of induction, a Hebbian neuron converging on what its inputs share, a learning curve timed by prediction error, and the capacity of a hippocampus, ending on the calcium concentration that induces LTP, the trials a rat needs and the patterns its CA3 can hold

Data: a spine of volume $0.08\,\mathrm{fL}$ ; each NMDA channel carries $10^{3}$ Ca $^{2+}$ per opening of $50\,\mathrm{ms}$ ; a synapse has $10$ NMDA receptors; $98\,\%$ of entering calcium is buffered; resting free calcium $0.1\,\text{µ}\mathrm{mol}/\mathrm{L}$ ; CaMKII is activated above $2\,\text{µ}\mathrm{mol}/\mathrm{L}$ free calcium, calcineurin between $0.3$ and $1\,\text{µ}\mathrm{mol}/\mathrm{L}$ . Hebb: two inputs with $C = \begin{pmatrix}1 & 0.6\\ 0.6 & 1\end{pmatrix}$ , $\eta = 0.1$ . Rescorla–Wagner: $\alpha = 0.15$ . Capacity: rat CA3 $3\times 10^{5}$ cells, $P = 0.14N$ ; a rat explores $20$ new places a day.

Part I — Calcium.

All ten NMDA receptors open once. How many calcium ions enter, how many stay free, and what concentration rise is that in the spine?
Is CaMKII activated? What if only two receptors open (glutamate released but the spine barely depolarised)?
With two receptors open, which enzyme is in its range, and what happens to the synapse? Explain how one ion can produce both LTP and LTD.
Calcium is pumped out with a time constant of $20\,\mathrm{ms}$ . Why must inputs arrive within tens of milliseconds to sum their calcium, and how does this set the spike-timing window?
A spine is $0.5\,\text{µ}\mathrm{m}$ from its neighbour on the dendrite. Explain why buffering and the spine’s narrow neck keep the calcium signal in one spine, and why that matters for input specificity.
The magnesium block is relieved at about $-30\,\mathrm{mV}$ . A single AMPA synapse depolarises the spine by $5\,\mathrm{mV}$ from $-70\,\mathrm{mV}$ . How many synchronous inputs (or what else) are needed to unblock NMDA receptors? What does this say about cooperativity?

Part II — A Hebbian neuron.

Find the eigenvalues and eigenvectors of $C$ .
Starting from $\mathbf{w} = (1, 0)$ , apply the averaged Hebb rule $\mathbf{w} \leftarrow \mathbf{w} + \eta C\mathbf{w}$ for three steps. What is the angle of $\mathbf{w}$ to the $(1,1)$ direction after each?
After many steps, what is the ratio $w_{2}/w_{1}$ , and what happens to $|\mathbf{w}|$ ?
Apply Oja’s averaged rule $\mathbf{w} \leftarrow \mathbf{w} + \eta\,(C\mathbf{w} - (\mathbf{w}^{\mathsf{T}}C\mathbf{w}) \mathbf{w})$ for one step from $(0.8, 0.8)$ . Toward what does it move?
The neuron now responds to the two inputs together. If input 2 alone fires (rare), what is $y$ relative to the joint case, with $\mathbf{w} = (0.71, 0.71)$ ?
Explain in one sentence how the associativity of LTP (a weak input paired with a strong one is strengthened) is this theorem in action.

Part III — Learning curve.

With $\alpha = 0.15$ , compute $V_{n}/\lambda$ after $5$ , $10$ and $20$ trials.
How many trials to reach $95\,\%$ of $\lambda$ ?
A rat learns a maze in about this many trials. If the reward is doubled, does the model predict faster learning in trials? What changes?
Cue A is trained to asymptote, then A and B are presented together with the same reward for $20$ trials. What is $V_{B}$ ? What do the dopamine neurons do on the compound trials?
The reward now follows B alone. Describe the first trial’s prediction error and dopamine response, and the course of $V_{B}$ .
A drug raises dopamine regardless of outcome. Using the rule, explain why the actions preceding the drug are learned as if rewarded, whatever their consequences.

Part IV — The hippocampus.

Hopfield capacity of the rat’s CA3.
At $20$ new places a day, how long before the capacity is reached? What must happen for the animal to go on learning?
Consolidation transfers memories to the cortex over weeks. Explain why a slow transfer protects old cortical memories (catastrophic interference) and why the hippocampus is a temporary store.
If $3\,\%$ of CA3 cells are active in any pattern, how many cells encode one place, and how many patterns could be distinguished if patterns were required to share fewer than half their cells? (Argue qualitatively why sparse coding raises capacity.)
Replay during sleep compresses a $10\,\mathrm{s}$ run into $0.5\,\mathrm{s}$ . If the spike-timing window is $20\,\mathrm{ms}$ , what separation between place cells’ firing in the run becomes Hebbian in the replay? Why might this be the point of compression?
The maze memory survives a hippocampal lesion made a month after training but not one made a day after. Interpret, with H.M.’s pattern of loss.
Summarise: the free calcium concentration after ten NMDA openings (question 1), the trials to $95\,\%$ learning (question 14), and the rat’s CA3 capacity (question 19).

Solution

Solution of Problem 19.1.

1. $10\times 10^{3} = 10^{4}$ ions; $2\,\%$ free, $200$ ; $200/6\times 10^{23} = 3.3\times 10^{-22}$ mol in $8\times 10^{-17}$ L: $4.2\,\text{µ}\mathrm{mol}/\mathrm{L}$ . 2. Yes, above the $2\,\text{µ}\mathrm{mol}/\mathrm{L}$ threshold. Two receptors: $40$ free ions, $0.83\,\text{µ}\mathrm{mol}/\mathrm{L}$ — below it. 3. At $0.83\,\text{µ}\mathrm{mol}/\mathrm{L}$ calcineurin is active and CaMKII is not: AMPA receptors are removed and the synapse weakens (LTD). One ion, two enzymes of different affinity: a large fast rise reaches the kinase, a small slow one only the phosphatase. 4. A second input within a few time constants adds its calcium to what remains of the first; after $50\,\mathrm{ms}$ almost nothing is left. The NMDA coincidence requirement and the $20\,\mathrm{ms}$ decay together give a window of some $20\,\mathrm{ms}$ on either side — the spike-timing window. 5. Buffers capture most calcium within a fraction of a micrometre and the narrow spine neck slows diffusion to the dendrite, so the rise stays in the spine that was active; a neighbour $0.5\,\text{µ}\mathrm{m}$ away sees nothing, and only the active synapse changes. 6. $(70 - 30)/5 = 8$ synchronous inputs, or a back-propagating action potential from the cell body: a single synapse cannot unblock its own NMDA receptors — cooperativity is built into the physics. 7. $(1,1)/\sqrt{2}$ with eigenvalue $1.6$ ; $(1,-1)/\sqrt{2}$ with $0.4$ . 8. $C\mathbf{w}_{0} = (1, 0.6)$ , $\mathbf{w}_{1} = (1.10, 0.06)$ , angle to $(1,1)$ : $45^{\circ} - 3.1^{\circ} = 41.9^{\circ}$ . $\mathbf{w}_{2} = (1.21, 0.13)$ : $38.8^{\circ}$ . $\mathbf{w}_{3} = (1.34, 0.22)$ : $35.8^{\circ}$ . 9. $w_{2}/w_{1} \to 1$ while $|\mathbf{w}|$ grows without bound (by a factor $1.16$ per step). 10. $C\mathbf{w} = (1.28, 1.28)$ ; $\mathbf{w}^{\mathsf{T}}C \mathbf{w} = 2.05$ ; $\Delta\mathbf{w} = 0.1\times(1.28 - 2.05\times 0.8) = -0.036$ each: $\mathbf{w} = (0.76, 0.76)$ , shrinking toward the unit vector $(0.71, 0.71)$ . 11. Joint $y = 1.42$ ; input 2 alone $y = 0.71$ , half. 12. The weak input fires while the strong one drives the cell, so it is correlated with the output and Hebb’s rule strengthens it — the neuron learns the co-occurrence. 13. $1 - 0.85^{n}$ : $0.56$ , $0.80$ , $0.96$ . 14. $0.85^{n} \le 0.05$ : $n \ge 18.5$ , so $19$ trials. 15. No: the fraction of asymptote reached after $n$ trials does not depend on $\lambda$ ; the asymptote doubles. A larger reward may raise $\alpha$ (salience), and a behavioural threshold on the absolute value is reached sooner. 16. $V_{B} = 0$ : blocked. The compound is fully predicted, the error is zero, dopamine neurons are silent. 17. B alone predicts nothing, so the error is $\lambda$ and dopamine bursts; $V_{B}$ then rises as in acquisition, $\lambda(1 - 0.85^{n})$ . 18. The drug delivers the burst that means “better than predicted” after whatever preceded it; the rule increments the value of those cues and actions on every occasion, so they acquire value independent of any real outcome — the seeking becomes compulsive. 19. $0.14\times 3\times 10^{5} \approx 4.2\times 10^{4}$ . 20. $4.2\times 10^{4}/20 = 2100$ days, about six years; consolidation must move memories to the cortex and forgetting must free the store. 21. Writing new patterns quickly into the cortex would overwrite the synapses encoding old ones (catastrophic interference); slow, interleaved replay lets the cortex integrate the new with the old, while the hippocampus holds the new memory meanwhile — a fast buffer feeding a slow store. 22. $0.03\times 3\times 10^{5} = 9000$ cells per place. Sparse patterns overlap little, so many more can be stored before their interference corrupts retrieval; capacity rises faster than $N$ as patterns become sparser, at the cost of each pattern using more cells’ worth of specificity. 23. Compression by $20$ : cells firing $400\,\mathrm{ms}$ apart in the run fire $20\,\mathrm{ms}$ apart in replay — inside the spike-timing window. The point of compression is to bring sequences too slow to be associated in real time within Hebbian reach. 24. At a day the memory still depends on the hippocampus; at a month it has been consolidated into the cortex and no longer needs it — exactly H.M.’s temporally graded loss, with recent memories gone and remote ones spared. 25. About $4.2\,\text{µ}\mathrm{mol}/\mathrm{L}$ of free calcium after ten openings; $19$ trials to $95\,\%$ ; some $4\times 10^{4}$ patterns in the rat’s CA3.

Terms defined in this chapter

See all 479 terms in the glossary