Hidden variable theory

In physics, a hidden variable theory is urged by a minority of physicists who argue that the statistical nature of quantum mechanics implies that quantum mechanics is incomplete; it is really applicable only to ensembles of particles; new physical phenomena beyond quantum mechanics are needed to explain an individual event.
Quantum mechanics is nondeterministic meaning that it generally does not predict the outcome of any measurement with certainty. Instead, it merely tells us what the probabilities of the outcomes are. This leads to the strange situation where measurements of a certain property done on two identical systems can give different answers. The question naturally arises whether there might be some deeper reality hidden beneath quantum mechanics, to be described by a more fundamental theory that can always predict the outcome of each measurement with certainty. An analogy exists with opinion polling: it is not that opinions are indefinite, but only if a reasonable sample of the population has been polled does one expect the poll results, as statistics, to be in line with the trend in the population at large.
In other words, quantum mechanics as it stands might be an incomplete description of reality. Some physicists maintain that underlying this level of indeterminacy there is an objective foundation. Such a theory is called a hidden variable theory. Most believe, however, that there is no deeper reality in quantum mechanics — that, indeed, experiments have shown hidden variables to be incompatible with observations.
In 1935, Einstein, Podolsky and Rosen wrote a four page paper called Can quantummechanical description of physical reality be considered complete? (http://prola.aps.org/pdf/PR/v47/i10/p777_1) that argued that such a theory was not only possible, but in fact necessary, proposing the EPR Paradox as proof. In 1964, John Bell showed, through his famous theorem with its Bell inequalities, that the kind of theory proposed by Einstein, Podolsky and Rosen made different experimental predictions than orthodox quantum mechanics.
Experiments have been interpreted as showing the orthodox account to be correct, but the hope for a socalled local hidden variable theory is still very much alive. The loopholes in experiments such as Aspect's are more serious than is generally realised.
A hiddenvariable theory, with its underlying determinism, which is consistent with quantum mechanics would have to be nonlocal, maintaining the existence of instantaneous causal relations between physically separated entities. Nonlocal theories, i.e. theories that allow systems to interact over distances with speeds greater than the speed of light, would not be ruled out. The first hiddenvariable theory was the pilot wave theory by Louis de Broglie from the late 1920s. The currently bestknown hiddenvariable theory, the Bohmian mechanics, of the physicist and philosopher David Bohm, created in 1952, is a nonlocal hidden variable theory.
The Bohm interpretation still enjoys a modest popularity among physicists, although most find it theoretically inelegant. However, there is no consensus. What Bohm did, based on an idea originally by de Broglie, was to posit both the quantum particle, e.g. an electron, and a hidden 'guiding wave' that governs its motion. Thus, in this theory electrons are quite clearly particles. When you perform a doubleslit experiment (see waveparticle duality), they go through one slit rather than the other. However, their choice of slit is not random but is governed by the guiding wave, resulting in the wave pattern that is observed.
Such a view contradicts the simple idea of local events that is used in both classical atomism and relativity theory. It points to a more holistic, mutually interpenetrating and interacting view of the world. Indeed Bohm himself stressed the holistic aspect of quantum theory in his later years, when he became interested in the ideas of J. Krishnamurti. The Bohm interpretation (as well as others) has also been the basis of some books which attempt to connect physics with Eastern mysticism and "consciousness".
The main weakness of Bohm's theory is that it looks contrived — which it is. It was deliberately designed to give predictions which are in all details identical to conventional quantum mechanics. His aim was not to make a serious counterproposal but simply to demonstrate that hiddenvariables theories are indeed possible. This actually was a significant breakthrough in thinking. His hope was that this could lead to new insights and experiments that would lead beyond the current quantum theories.
Another type of deterministic theory [1] was recently introduced by Gerard 't Hooft. This theory is motivated by the problems that are encountered when one tries to formulate a unified theory of quantum gravity.
Most physicists however are of the position that the true theory of the universe is not a hidden variable theory and that particles do not have any extra information which is not present in their quantum mechanics description. These other interpretations of quantum mechanics have their own philosophical issues. A very small number of physicists believe that local realism is correct and that quantum mechanics is ultimately incorrect.
References
[1] Gerard 't Hooft, Quantum Gravity as a Dissipative Deterministic System, Class. Quant. Grav. 16, 32633279 (1999) preprint (http://xxx.lanl.gov/abs/grqc/9903084).