Subsections

• 10.2.1 A Mathematical Formulation of Hebb's Rule
  • 10.2.1.1 Example: Postsynaptic gating versus presynaptic gating
  • 10.2.1.2 Example: Covariance rule
  • 10.2.1.3 Example: Quadratic terms
  • 10.2.1.4 Example: Bienenstock-Cooper-Munroe rule

10.2 Rate-Based Hebbian Learning

In order to prepare the ground for a thorough analysis of spike-based learning rules in Section 10.3 we will first review the basic concepts of correlation-based learning in a firing rate formalism.

10.2.1 A Mathematical Formulation of Hebb's Rule

In order to find a mathematically formulated learning rule based on Hebb's postulate we focus on a single synapse with efficacy w_ij that transmits signals from a presynaptic neuron j to a postsynaptic neuron i. For the time being we content ourselves with a description in terms of mean firing rates. In the following, the activity of the presynaptic neuron is denoted by $\nu_{j}^{}$ and that of the postsynaptic neuron by $\nu_{i}^{}$.

There are two aspects in Hebb's postulate that are particularly important, viz. locality and cooperativity. Locality means that the change of the synaptic efficacy can only depend on local variables, i.e., on information that is available at the site of the synapse, such as pre- and postsynaptic firing rate, and the actual value of the synaptic efficacy, but not on the activity of other neurons. Based on the locality of Hebbian plasticity we can make a rather general ansatz for the change of the synaptic efficacy,

$${{\text{d}}\over {\text{d}}t} \nu_{i}^{} \nu_{j}^{}$$ (10.1)

Here, dw_ij/dt is the rate of change of the synaptic coupling strength and F is a so far undetermined function (Brown et al., 1991; Kohonen, 1984; Sejnowski and Tesauro, 1989). We may wonder whether there are other local variables (e.g., the membrane potential u_i) that should be included as additional arguments of the function F. It turns out that in standard rate models this is not necessary, since the membrane potential u_i is uniquely determined by the postsynaptic firing rate, $\nu_{i}^{}$ = g(u_i), with a monotone gain function g.

The second important aspect of Hebb's postulate, cooperativity, implies that pre- and postsynaptic neuron have to be active simultaneously for a synaptic weight change to occur. We can use this property to learn something about the function F. If F is sufficiently well-behaved, we can expand F in a Taylor series about $\nu_{i}^{}$ = $\nu_{j}^{}$ = 0,

$${{\text{d}}\over {\text{d}}t} \nu_{i}^{} \nu_{j}^{}$$

$$\nu_{j}^{2} \nu_{{i}}^{2} \nu_{i}^{} \nu_{j}^{} \mathcal {O} \nu^{3}_{}$$ (10.2)

The term containing c^corr₂ on the right-hand side of (10.2) is bilinear in pre- and postsynaptic activity. This term implements the AND condition for cooperativity which makes Hebbian learning a useful concept.

The simplest choice for our function F is to fix c^corr₂ at a positive constant and to set all other terms in the Taylor expansion to zero. The result is the prototype of Hebbian learning,

$${{\text{d}}\over {\text{d}}t} \nu_{i}^{} \nu_{j}^{}$$ (10.3)

We note in passing that a learning rule with c^corr₂ < 0 is usually called anti-Hebbian because it weakens the synapse if pre- and postsynaptic neuron are active simultaneously; a behavior that is just contrary to that postulated by Hebb. A learning rule with only first-order terms gives rise to so-called non-Hebbian plasticity, because pre- or postsynaptic activity alone induces a change of the synaptic efficacy. More complicated learning rules can be constructed if higher-order terms in the expansion of Eq. (10.2), such as $\nu_{i}^{}$ $\nu_{j}^{2}$, $\nu_{i}^{2}$ $\nu_{j}^{}$, $\nu_{i}^{2}$ $\nu_{j}^{2}$, etc., are included.

The dependence of F on the synaptic efficacy w_ij is a natural consequence of the fact that w_ij is bounded. If F was independent of w_ij then the synaptic efficacy would grow without limit if the same potentiating stimulus is applied over and over again. A saturation of synaptic weights can be achieved, for example, if the parameter c^corr₂ in Eq. (10.2) tends to zero as w_ij approaches its maximum value, say w^max = 1, e.g.,

$$\gamma_{2}^{}$$ (10.4)

with a positive constant $\gamma_{2}^{}$.

Hebb's original proposal does not contain a rule for a decrease of synaptic weights. In a system where synapses can only be strengthened, all efficacies will finally saturate at their upper maximum value. An option of decreasing the weights (synaptic depression) is therefore a necessary requirement for any useful learning rule. This can, for example, be achieved by weight decay, which can be implemented in Eq. (10.2) by setting

$$\gamma_{0}^{}$$ (10.5)

Here, $\gamma_{0}^{}$ is (small) positive constant that describes the rate by which w_ij decays back to zero in the absence of stimulation. Our formulation (10.2) is hence sufficiently general to allow for a combination of synaptic potentiation and depression. If we combine Eq. (10.4) and Eq. (10.5) we obtain the learning rule

$${{\text{d}}\over {\text{d}}t} \gamma_{2}^{} \nu_{i}^{} \nu_{j}^{} \gamma_{0}^{}$$ (10.6)

The factors (1 - w_ij) and w_ij that lead to a saturation at w_ij = 1 for continued stimulation and an exponential decay to w_ij = 0 in the absence of stimulation, respectively, are one possibility to implement `soft' bounds for the synaptic weight. In simulations, `hard' bounds are often used to restrict the synaptic weights to a finite interval, i.e., a learning rule with weight-independent parameters is only applied as long as the weight stays within its limits.

Another interesting aspect of learning rules is competition. The idea is that synaptic weights can only grow at the expense of others so that if a certain subgroup of synapses is strengthened, other synapses to the same postsynaptic neuron have to be weakened. Competition is essential for any form of self-organization and pattern formation. Practically, competition can be implemented in simulations by normalizing the sum of all weights converging onto the same postsynaptic neuron (Miller and MacKay, 1994); cf. Section 11.1.3. Though this can be motivated by a limitation of common synaptic resources such a learning rule violates locality of synaptic plasticity. On the other hand, competition of synaptic weight changes can also be achieved with purely local learning rules (Kistler and van Hemmen, 2000a; Kempter et al., 2001; Oja, 1982; Song et al., 2000).

10.2.1.1 Example: Postsynaptic gating versus presynaptic gating

Equation (10.6) is just one possibility to specify rules for the growth and decay of synaptic weights. In the framework of Eq. (10.2), other formulations are conceivable; cf. Table 10.1. For example, we can define a learning rule of the form

$${{\text{d}}\over {\text{d}}t} \gamma \nu_{i}^{} \nu_{j}^{} \nu_{\theta}^{}$$ (10.7)

where $\gamma$ is a positive constant and $\nu_{\theta}^{}$ is some reference value that may depend on the current value of w_ij. A weight change occurs only if the postsynaptic neuron is active, $\nu_{i}^{}$ > 0. We say that weight changes are `gated' by the postsynaptic neuron. The direction of the weight change depends on the sign of the expression in the rectangular brackets. Let us suppose that the postsynaptic neuron is driven by a subgroup of highly active presynaptic neurons ($\nu_{i}^{}$ > 0 and $\nu_{j}^{}$ > $\nu_{\theta}^{}$). Synapses that connect the postsynaptic neuron and one of the highly active presynaptic neurons are strengthened while the efficacy of other synapses that have not been activated is decreased. Firing of the postsynaptic neuron thus leads to LTP at the active pathway (`homosynaptic LTP') and at the same time to LTD at the inactive synapses (`heterosynaptic LTD'); for reviews see Linden and Connor (1995); Brown et al. (1991); Bi and Poo (2001).

A particularly interesting case from a theoretical point of view is the choice $\nu_{\theta}^{}$(w_ij) = w_ij, i.e.,

$${{\text{d}}\over {\text{d}}t} \nu_{i}^{} \nu_{j}^{}$$ (10.8)

The synaptic weights thus approach the fixed point w_ij = $\nu_{j}^{}$ whenever the postsynaptic neuron is active. In the stationary state, the set of weight values w_ij reflects the presynaptic firing pattern $\nu_{j}^{}$, 1$\le$j$\le$N. In other words, the presynaptic firing pattern is stored in the weights. This learning rule is an important ingredient of competitive unsupervised learning (Grossberg, 1976; Kohonen, 1984).

Let us now turn to a learning rule where synaptic changes are `gated' by the presynaptic activity $\nu_{j}^{}$. The corresponding equation has the same form as Eq. (10.7) except that the role of pre- and postsynaptic firing rate are exchanged,

$${{\text{d}}\over {\text{d}}t} \gamma \nu_{i}^{} \nu_{\theta}^{} \nu_{j}^{}$$ (10.9)

In this case, a change of synaptic weights can only occur if the presynaptic neuron is active ($\nu_{j}^{}$ > 0). The direction of the change is determined by the activity of the postsynaptic neuron. For $\gamma$ > 0, the synapse is strengthened if the postsynaptic cell is highly active ( $\nu_{i}^{}$ > $\nu_{\theta}^{}$); otherwise it is weakened.

For $\gamma$ < 0, the correlation term has a negative sign and the learning rule (10.9) gives rise to anti-Hebbian plasticity, which has an interesting stabilizing effect on the postsynaptic firing rate. If the presynaptic firing rates are kept constant, the postsynaptic firing rate $\nu_{i}^{}$ will finally converge to the reference value $\nu_{\theta}^{}$. To see why let us consider a simple rate neuron with output rate $\nu_{i}^{}$ = g($\sum_{j}^{}$w_ij$\nu_{j}^{}$). For $\nu_{i}^{}$ < $\nu_{\theta}^{}$, all synapses are strengthened ( dw_ij/dt > 0 for all j) and the overall input strength h_i = $\sum_{j}^{}$w_ij$\nu_{j}^{}$ is increasing. Since g is a monotonously growing function of h_i, the output rate tends to $\nu_{\theta}^{}$. On the other hand, if $\nu_{i}^{}$ > $\nu_{\theta}^{}$, all synaptic efficacies decrease and so does $\nu_{i}^{}$. Hence, $\nu_{i}^{}$ = $\nu_{\theta}^{}$ is a globally attractive fixed point of the postsynaptic activity. Some of the detailed spike-based learning rule, to be discussed below, will show a similar stabilization of the postsynaptic activity.

10.2.1.2 Example: Covariance rule

Sejnowski and Tesauro (1989) have suggested a learning rule of the form

$${{\rm d}\over {\rm dt}} \gamma \left(\vphantom{ \nu_i- \langle \nu_i \rangle }\right. \nu_{i}^{} \langle \nu_{i}^{} \rangle \left.\vphantom{ \nu_i- \langle \nu_i \rangle }\right) \left(\vphantom{ \nu_j-\langle \nu_j \rangle}\right. \nu_{j}^{} \langle \nu_{j}^{} \rangle \left.\vphantom{ \nu_j-\langle \nu_j \rangle}\right)$$ (10.10)

called covariance rule. This rule is based on the idea that the rates $\nu_{i}^{}$(t) and $\nu_{j}^{}$(t) fluctuate around mean values $\langle$$\nu_{i}^{}$$\rangle$,$\langle$$\nu_{j}^{}$$\rangle$ that are taken as running averages over the recent firing history. To allow a mapping of the covariance rule to the general framework of Eq. (10.2), the mean firing rates $\langle$$\nu_{i}^{}$$\rangle$ and $\langle$$\nu_{j}^{}$$\rangle$ have to be constant in time. We will return to the covariance rule in Chapter 11.

Table 10.1: The change ${{\text{d}}\over {\text{d}}t}$w_ij of a synapse from j to i for various Hebb rules as a function of pre- and postsynaptic activity. `ON' indicates a neuron firing at high rate ($\nu$ > 0), whereas `OFF' means an inactive neuron ($\nu$ = 0). From left to right: Standard Hebb rule, Hebb with decay, pre- and postsynaptic gating, covariance rule. The parameters are 0 < $\nu_{\theta}^{}$ < $\nu^{{\rm max}}_{}$ and 0 < c₀ < ($\nu^{{\rm max}}_{}$)².

10.2.1.3 Example: Quadratic terms

All of the above learning rules had c^pre₂ = c^post₂ = 0. Let us now consider a nonzero quadratic term c^post₂ = - $\gamma$ w_ij. We take c^corr₂ = $\gamma$ > 0 and set all other parameters to zero. The learning rule

$${{\text{d}}\over {\text{d}}t} \gamma \nu_{i}^{} \nu_{j}^{} \nu_{i}^{2}$$ (10.11)

is called Oja's rule (Oja, 1982). As we will see in Chapter 11.1.3, Oja's rule converges asymptotically to synaptic weights that are normalized to $\sum_{j}^{}$w_ij² = 1 while keeping the essential Hebbian properties of the standard rule of Eq. (10.3). We note that normalization of $\sum_{j}^{}$w_ij² implies competition between the synapses that make connections to the same postsynaptic neuron, i.e., if some weights grow others must decrease.

10.2.1.4 Example: Bienenstock-Cooper-Munroe rule

Higher terms in the expansion on the right-hand side of Eq. (10.2) lead to more intricate plasticity schemes. As an example, let us consider a generalization of the presynaptic gating rule in Eq. (10.9)

$${{\text{d}}\over {\text{d}}t} \eta \phi \nu_{i}^{} \nu_{\theta}^{} \nu_{j}^{} \gamma$$ (10.12)

with a nonlinear function $\phi$ and a reference rate $\nu_{\theta}^{}$. If we replace $\nu_{\theta}^{}$ by a running average of the output rate, $\langle$$\nu_{i}^{}$$\rangle$, then we obtain the so-called Bienenstock-Cooper-Munroe (BCM) rule (Bienenstock et al., 1982).

Some experiments (Artola et al., 1990; Ngezahayo et al., 2000; Artola and Singer, 1993) suggest that the function $\phi$ should look similar to that sketched in Fig. 10.5. Synaptic weights do not change as long as the postsynaptic activity stays below a certain minimum rate, $\nu_{0}^{}$. For moderate levels of postsynaptic excitation, the efficacy of synapses activated by presynaptic input is decreased. Weights are increased only if the level of postsynaptic activity exceeds a second threshold, $\nu_{\theta}^{}$. The change of weights is restricted to those synapses which are activated by presynaptic input, hence the `gating' factor $\nu_{j}^{}$ in Eq. (10.12). By arguments completely analogous to the ones made above for the presynaptically gated rule, we can convince ourselves that the postsynaptic rate has a fixed point at $\nu_{\theta}^{}$. For the form of the function shown in Fig.10.5 this fixed point is unstable. In order to avoid that the postsynaptic firing rate blows up or decays to zero, it is therefore necessary to turn $\nu_{\theta}^{}$ into an adaptive variable (Bienenstock et al., 1982). We will come back to the BCM rule towards the end of this chapter.

Figure 10.5: Bidirectional learning rule. Synaptic plasticity is characterized by two thresholds for the postsynaptic activity (Bienenstock et al., 1982). Below $\nu_{0}^{}$ no synaptic modification occurs, between $\nu_{0}^{}$ and $\nu_{\theta}^{}$ synapses are depressed, and for postsynaptic firing rates beyond $\nu_{\theta}^{}$ synaptic potentiation can be observed. A similar dependence is found if weight changes are plotted as a function of the postsynaptic potential rather than the postsynaptic rate (Ngezahayo et al., 2000; Artola and Singer, 1993).