Learning with bounded synapses
generates synaptic democracy and balanced neurons
Computational Neuroscience Physiology University of Berne Buehlplatz 5 |
phone: +41 31 631 87 21 email: wsenn@cns.unibe.ch www: http://www.cns.unibe.ch/~wsenn/biol_cyb_index.html |
Abstract:
Learning in a neuronal network is often thought of as
a linear superposition of synaptic modifications induced by individual stimuli.
However, since biological synapses are naturally bounded, a linear
superposition would cause fast forgetting of previously acquired memory. This
represents a problem for any type of learning rule, including STDP. It will be
shown that forgetting due to synaptic saturation can be avoided by additional
simple constraints. As an example I present the classical perceptron learning
problem. In this example, synapses are modified only if the postsynaptic
response does not match the desired output. I show that the original memory
capacity with unbounded weights is regained, provided there is (1) some global
inhibition, (2) a small learning rate, and (3) a small neuronal threshold.
Simulations suggest that these conditions are also necessary. The maximal
storage capacity is also reestablished if the synapses are distributed over a
spatially extended dendritic tree, provided that distal synapses are allowed to
attain stronger weights. After successful learning, excitation will roughly
balance inhibition. Moreover, learning a large number of patterns urges the
synapses to acquire similar strengths when measured in the soma. The fact that
synapses saturate has the additional benefit that non-separable patterns, e.g.
similar patterns with contradicting outputs, eventually generate a subthreshold
response, and therefore suppress neurons which can not provide any information.
Hence, synaptic equalization, neuronal balancing, and neuronal suppression can
be view as a consequence of successfully dealing with bounded synapses.
Spike-based synaptic plasticity and the emergence of
direction selective simple cells: mathematical analysis. W. Senn and N.J.
Buchs, Journal of Computational Neuroscience, 14:119-138 (2003).
http://www.cns.unibe.ch/publications/ftp/paper_anadirselect.pdf.
Spike-based synaptic plasticity and the emergence of
direction selective simple cells: simulation results. N.J. Buchs and W. Senn,
Journal of Computational Neuroscience, 13:167-186 (2002).
http://www.cns.unibe.ch/publications/ftp/paper_simdirselect_JCNS.pdf
Activity-dependent development of axonal and dendritic
delays or, why synaptic transmission should be unreliable. W. Senn, M.
Schneider and B. Ruf, Neural Computation 14(3), 583-620 (2002).
http://www.cns.unibe.ch/publications/ftp/paper_DelayNECO.pdf
Beyond spike-timing: the role of nonlinear plasticity
and unreliable synapses. W. Senn, Biological Cybernetics, 87:344-355 (2002).
Review of the above works.
http://www.cns.unibe.ch/publications/ftp/paper_synalgo2.pdf
An algorithm for modifying neurotransmitter release probability based on pre- and post-synaptic spike timing. W. Senn, H. Markram and M. Tsodyks, Neural Computation 13(1), 35-68 (2001). http://www.cns.unibe.ch/publications/ftp/paper_Synalgo.pdf
For a collection of theoretical and experimental
reviews on STDP see "Hebb in Perspective", Special issue of
Biological Cybernetics, vol. 87, issues
5-6, 2002,
http://link.springer.de/link/service/journals/00422/tocs/t2087005.htm.