Perceptron
|
|
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt.
The perceptron consists of one or more layers of artificial neurons; the inputs are fed directly to the outputs via a series of weights. In this way it can be considered the simplest kind of feedforward network. Each neuron calculates a weighted sum of its inputs - that is, the sum for all inputs of the product of an input and its corresponding weight. If this value is above some threshold, the neuron is said to 'fire', outputting the value 1; otherwise it takes the value -1. To simplify training, the threshold is often represented as an extra weight attached to a constant input, with the actual threshold function centred on 0.
More generally, after the sum of the previous layer times the weights is computed for each neuron, it is passed through a nonlinearity function. The sigmoid function is a popular choice, because of its simple derivative. The nonlinearity function is necessary for multilayer networks, because otherwise they are linear and equivalent to simple two-layer perceptrons.
Artificial neurons with this kind of activation function are also called McCulloch-Pitts neurons or threshold neurons. In the literature the term perceptron sometimes also refers to networks consisting of just one of these units. Perceptrons can be trained by a simple learning algorithm that is usually called the delta rule. It calculates the errors between calculated output and sample output data, and uses this to create an adjustment to the weights, thus implementing a form of gradient descent.
Although the perceptron initially seemed promising, it was quickly proved that simple perceptrons could not be trained to recognise many classes of patterns. This led to the field of neural network research stagnating for many years, before it was recognised that neural networks with three or more layers had far greater processing power than simpler perceptrons.
Simple perceptrons with one or two layers are only capable of learning linearly separable patterns; in 1969 a famous monograph entitled Perceptrons by Marvin Minsky and Seymour Papert showed that it was impossible for these classes of network to learn an XOR function. They conjectured (incorrectly) that a similar result would hold for a perceptron with three or more layers.
The discovery in the 1980s that multi-layer neural networks did not, in fact, have these problems led to the resurgence of neural network research.
See also: linear classifier
References
- Rosenblatt, Frank (1958), The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Cornell Aeronautical Laboratory, Psychological Review, v65, No. 6, pp. 386-408.
- Minsky M L and Papert S A 1969 Perceptrons (Cambridge, MA: MIT Press)
- Widrow, B., Lehr, M.A., "30 years of Adaptive Neural Networks: Peceptron, Madaline, and Backpropagation," Proc. IEEE, vol 78, no 9, pp. 1415-1442, (1990).
See also
- Artificial intelligence
- Artificial neural network
- Data mining
- Linear classifier
- Linear discriminant analysis
- Logit (for Logistic Regression)
- Machine learning
- Pattern recognition
External links
- History of perceptrons (http://www.csulb.edu/~cwallis/artificialn/History.htm)
- Mathematics of perceptrons (http://www.cis.hut.fi/ahonkela/dippa/node41.html)
- Perceptron demo applet (http://diwww.epfl.ch/mantra/tutorial/english/perceptron/html/)
"Perceptron" is also the name of a Michigan company that sells technology products to automakers.de:Perzeptron pl:Perceptron th:เพอร์เซปตรอน