Statistical Mechanics of Neural Networks

DEC 01, 1988

Studies of disordered systems have generated new insights into the cooperative behavior and emergent computational properties of large, highly connected networks of simple, neuron‐like processors.

DOI: 10.1063/1.881142

Haim Sompolinsky

A neural network is a large, highly interconnected assembly of simple elements. The elements, called neurons, are usually two‐state devices that switch from one state to the other when their input exceeds a specific threshold value. In this respect the elements resemble biological neurons, which fire—that is, send a voltage pulse down their axons—when the sum of the inputs from their synapses exceeds a “firing” threshold. Neural networks therefore serve as models for studies of cooperative behavior and computational properties of the sort exhibited by the nervous system.

References

1. W. S. McCulloch, W. A. Pitts, Bull. Math. Biophys. 5, 115 (1943).https://doi.org/BMBIAO
2. D. O. Hebb, The Organization of Behavior, Wiley, New York (1949).
3. F. Rosenblatt, Principles of Neurodynamics, Spartan, Washington, D.C. (1961).
M. Minsky, S. Papert, Perceptrons, MIT P., Cambridge, Mass. (1988).
4. B. Widrow, in Self‐Organizing Systems, M. C. Yovits, G. T. Jacobi, G. D. Goldstein, eds., Spartan, Washington, D.C. (1962).
5. S. Amari, K. Maginu, Neural Networks 1, 63 (1988), and references therein.https://doi.org/NNETEB
6. S. Grossberg, Neural Networks 1, 17 (1988), and references therein.https://doi.org/NNETEB
7. T. Kohonen, Self Organization and Associative Memory, Springer‐Verlag, Berlin (1984).
T. Kohonen, Neural Networks 1, 3 (1988).https://doi.org/NNETEB
8. W. A. Little, Math. Biosci. 19, 101 (1974). https://doi.org/MABIAR
W. A. Little, G. L. Shaw, Math. Biosci. 39, 281 (1978).https://doi.org/MABIAR
9. J. J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554 (1982). https://doi.org/PNASA6
J. J. Hopfield, D. W. Tank, Science 233, 625 (1986), and references therein.https://doi.org/SCIEAS
10. M. Mezard, G. Parisi, M. A. Virasoro, Spin Glass Theory and Beyond, World Scientific, Singapore (1987).
K. Binder, A. P. Young, Rev. Mod. Phys. 58, 801 (1986).https://doi.org/RMPHAT
11. V. Braitenberg, in Brain Theory, G. Palm, A. Aertsen, eds., Springer‐Verlag, Berlin (1986), p. 81.
12. K. Binder, ed., Applications of the Monte Carlo Methods in Statistical Physics, Springer‐Verlag, Berlin (1984).
13. D. E. Rumelhart, J. L. McClell and the PDP Group, Parallel Distributed Processing, MIT P., Cambridge, Mass. (1986).
14. D. Kleinfeld, H. Sompolinsky, Biophys. Jour., in press, and references therein.
15. S. R. Kelso, A. H. Ganong, T. H. Brown, Proc. Natl. Acad. Sci. USA 83, 5326 (1986). https://doi.org/PNASA6
G. V. diPrisco, Prog. Neurobiol. 22, 89 (1984).
16. D. J. Amit, H. Gutfreund, H. Sompolinsky, Phys. Rev. A 32, 1007 (1985). https://doi.org/PLRAAN
D. J. Amit, H. Gutfreund, H. Sompolinsky, Ann. Phys. N. Y. 173, 30 (1987), and references therein.https://doi.org/APNYA6
17. M. Abeles, Local Cortical Circuits, Springer‐Verlag Berlin (1982).
18. D. J. Willshaw, O. P. Buneman, H. C. Longuet‐Higgins, Nature 222, 960 (1969). https://doi.org/NATUAS
A. Moopen, J. Lambe, P. Thakoor, IEEE SMC 17, 325 (1987).
19. E. Gardner, J. Phys. A 21, 257 (1988). https://doi.org/JPHAC5
A. D. Bruce, A. Canning, B. Forrest, E. Gardner, D. J. Wallace, in Neural Networks for Computing, J. S. Denker, ed., AIP, New York (1986) p. 65.
20. A. Crisanti, H. Sompolinsky, Phys. Rev. A 37, 4865 (1988).https://doi.org/PLRAAN
21. S. Kirkpatrick, C. D. GellatJr., M. P. Vecchi, Science 220, 671 (1983).https://doi.org/SCIEAS

More about the Authors

Haim Sompolinsky. Racah Institute of Physics, Hebrew University, Jerusalem.