Learning with bounded synapses generates synaptic democracy and balanced neurons

 

 

Walter Senn, in collaboration with Stefano Fusi

Computational Neuroscience Physiology
Institute of Physiology

University of Berne

Buehlplatz 5
3012 Bern
Switzerland

phone: +41 31 631 87 21

email: wsenn@cns.unibe.ch

www: http://www.cns.unibe.ch/~wsenn/biol_cyb_index.html

 

 

Abstract:

Learning in a neuronal network is often thought of as a linear superposition of synaptic modifications induced by individual stimuli. However, since biological synapses are naturally bounded, a linear superposition would cause fast forgetting of previously acquired memory. This represents a problem for any type of learning rule, including STDP. It will be shown that forgetting due to synaptic saturation can be avoided by additional simple constraints. As an example I present the classical perceptron learning problem. In this example, synapses are modified only if the postsynaptic response does not match the desired output. I show that the original memory capacity with unbounded weights is regained, provided there is (1) some global inhibition, (2) a small learning rate, and (3) a small neuronal threshold. Simulations suggest that these conditions are also necessary. The maximal storage capacity is also reestablished if the synapses are distributed over a spatially extended dendritic tree, provided that distal synapses are allowed to attain stronger weights. After successful learning, excitation will roughly balance inhibition. Moreover, learning a large number of patterns urges the synapses to acquire similar strengths when measured in the soma. The fact that synapses saturate has the additional benefit that non-separable patterns, e.g. similar patterns with contradicting outputs, eventually generate a subthreshold response, and therefore suppress neurons which can not provide any information. Hence, synaptic equalization, neuronal balancing, and neuronal suppression can be view as a consequence of successfully dealing with bounded synapses.

 

Spike-based synaptic plasticity and the emergence of direction selective simple cells: mathematical analysis. W. Senn and N.J. Buchs, Journal of Computational Neuroscience, 14:119-138 (2003). http://www.cns.unibe.ch/publications/ftp/paper_anadirselect.pdf.

Spike-based synaptic plasticity and the emergence of direction selective simple cells: simulation results. N.J. Buchs and W. Senn, Journal of Computational Neuroscience, 13:167-186 (2002). http://www.cns.unibe.ch/publications/ftp/paper_simdirselect_JCNS.pdf

Activity-dependent development of axonal and dendritic delays or, why synaptic transmission should be unreliable. W. Senn, M. Schneider and B. Ruf, Neural Computation 14(3), 583-620 (2002). http://www.cns.unibe.ch/publications/ftp/paper_DelayNECO.pdf

Beyond spike-timing: the role of nonlinear plasticity and unreliable synapses. W. Senn, Biological Cybernetics, 87:344-355 (2002). Review of the above works. http://www.cns.unibe.ch/publications/ftp/paper_synalgo2.pdf

An algorithm for modifying neurotransmitter release probability based on pre- and post-synaptic spike timing. W. Senn, H. Markram and M. Tsodyks, Neural Computation 13(1), 35-68 (2001). http://www.cns.unibe.ch/publications/ftp/paper_Synalgo.pdf

For a collection of theoretical and experimental reviews on STDP see "Hebb in Perspective", Special issue of Biological Cybernetics, vol. 87,  issues 5-6, 2002, http://link.springer.de/link/service/journals/00422/tocs/t2087005.htm.