Talk:Bayes' theorem

Hello everyone. There was a recent substantial revision of Bayes' theorem [1] (http://en.wikipedia.org/w/wiki.phtml?title=Bayes%27_theorem&diff=4022419&oldid=4021020). I'm afraid it doesn't look like an improvement to me. Here are some points to consider.

1. In the introduction, Bayes' theorem is described in terms of random variables. This isn't necessary to clarify Bayes' theorem, and introduces a whole raft of heavy baggage (to mix metaphors) that is going to be well-nigh incomprehensible to the general readership.
2. The two-line derivation of Bayes' theorem is put off for several paragraphs by a lengthy digression which introduces some unnecessary notation and includes a verbal statement of Bayes' theorem which is much less clear than the algebraic statement which it displaces.
3. The example is somewhat problematic. It is formally correct, but it's not very compelling, as it doesn't make use of any relevant prior information; the medical test example and even the cookies example, which were moved to Bayesian inference a while ago, were superior in that respect. Perhaps if an example is needed, we can restore the medical test or cookies (I vote for the medical test fwiw). The example is also misplaced (coming before the algebraic statement) although that's easy to remedy.

Given the difficulties of the recent revision, I'm tempted to revert. Perhaps someone wants to talk me out of it. Regards, Wile E. Heresiarch 22:39, 11 Jul 2004 (UTC)

Perhaps you are right that the really simple material should come first. However, that's not a reason to throw away the example on political opinion polling. That example is in many respects typical of the simplest applications in Bayesian statistical inference. I for one find it compelling for that reason. To say that the simple statement that is followed by the words "... which is Bayes' theorem" is more than just a simple special case is misleading. Michael Hardy 23:41, 11 Jul 2004 (UTC)

Also, the "verbal" version is very useful; in some ways it makes a simple and memorable idea appear that is less-than-clearly expressed by the formula expressed in mathematical notation. The role of the likelihood and the role of the prior are extremely important ideas. Michael Hardy 23:44, 11 Jul 2004 (UTC)

I've moved the example farther down in the article, as it interrupts the exposition. I've also reverted the section "Statement of Bayes' theorem" to its previous form; the newer version did not introduce any new material, and was less clear. I put a paraphrase using the words posterior, prior, likelihood, & normalizing constant into the "Statement" section. -- I'm still not entirely happy with "random variable" in the introduction, but I haven't found a suitable replacement. I'd favor "proposition" but that it is likely not familiar to general readers. Fwiw & happy editing, Wile E. Heresiarch 14:49, 20 Jul 2004 (UTC)

---

Hello, I've moved the existing content of this page (last edit April 12, 2004) to Talk:Bayes' theorem/Archive1. I used the "move" function (instead of cut-n-paste) so the edit history is now with the archive page. Regards, Wile E. Heresiarch 14:30, 8 Jul 2004 (UTC)

Bayes' theorem vs Bayesian inference

It seems to me that the current version of the Bayes' theorem article contains a little too much Bayesian inference. This is not to deny from the importance of Bayesian inference as the premier application of Bayes' theorem, but as far as I can see:

1. The section explaining terms such as posterior, likelihood, etc. is more appropriate to the Bayesian inference article. None of it is taught with Bayes' theorem in courses on elementary probability (unless, I assume, Bayesian inference is also taught).
2. The example is one of Bayesian inference, not simply Bayes' theorem. Somewhat ironically, the Bayesian inference article contains some simple examples of Bayes' Theorem that are not Bayesian in nature, and that were moved there from an older version of the Bayes' theorem article!

Some of these things are noted in other posts to this talk page and the talk page of the Bayesian inference article, but I can't see that the current version of either article is a satisfactory outcome of the discussions. The current versions of the articles appear to muddy the distinction between Bayes' theorem and Bayesian inference/probability.

Hence, I propose to change these articles by

1. swapping the cookie jar and false positive examples from the Bayesian inference article for the example from the Bayes' theorem article;
2. deleting the section on conventional names of terms in the theorem from the Bayes' theorem article (but noting that there are such conventions as detailed in the Bayesian inference article);
3. revising the description of the theorem to refer to probabilities of events, since this is the most elementary way of expressing Bayes' theorem, and is consistent with identities given in (for instance) the conditional probability article.

Since this has been a topic of some discussion on the talk pages of both articles, I would like to invite further comment from others before I just go ahead and make these changes. In the absence of such discussion, I'll make the proposed changes in a few days.

Cheers, Ben Cairns 07:55, 23 Jan 2005 (UTC).

Well, I agree the present state of affairs isn't entirely satisfactory. About (1), if you want to move the medical test to Bayes' theorem in exchange for the voters example, I'm OK with that. I'd rather not clutter up Bayes' theorem with the cookies; it's no less complicated than the medical test, and a lot less interesting. (2) I'm OK with cutting the conventional terms from Bayes' theorem . (3) I guess I'm not entirely happy with stating Bayes' theorem as a theorem about events, since "events" has some baggage. I'd be happiest to say something like P(B|A) = P(A|B) P(B)/P(A) whenever A and B are objects for which P(A), P(B), etc, make sense and that might be OK for mathematically-minded readers but maybe not as friendly to the general readership. Any other thoughts about that? Anyway, thanks for reopening the discussion. Now that we've all had several months to think about, I'm sure we'll make quick progress. 8^) Regards & happy editing, Wile E. Heresiarch 21:56, 23 Jan 2005 (UTC)

Thanks for the quick response! I also prefer the medical test example. Perhaps the cookies can be returned home and then deleted. It's not so complicated a theorem that it needs many examples.

I also take your point about events, but it's just that event has a particular meaning. Perhaps a brief, layman's definition would be appropriate, for example:

"Bayes' theorem is a result in probability theory, which gives the conditional probability of an event (an outcome to which we may assign a probability) A given another event B in terms of the conditional probability of B given A and the (marginal) probabilities of A and B alone."

I don't believe this is a foolish consistency; a precise definition of an event is an important component of elementary probability theory, and anyone who would study the area (even in the kind of detail provided by Wikipedia) should come to appreciate that we cannot go around assigning probabilities to just anything. The article Event (probability theory) explains this quite well. It seems to me that the greater danger lies in obscuring the concept with an array of vaguer terms for which we do not have articles explaining the matter. Thanks again, Ben Cairns 22:43, 23 Jan 2005 (UTC).

Well, we seem to have reached an impasse. I'm quite aware that "event" has a prescribed meaning; that's why I want to omit it from article. Technical difficulties with strange sets never arise in practical problems and for this reason are at most a curiosity -- this is the pov of Jaynes the uber-Bayesian. From what I can tell, Bayesians are in fact happy to assign probability to "just anything" and this is pretty much the defining characteristic of their school. Let me see if I can find some textbook statements from Bayesians to see what is permitted for A and B. Wile E. Heresiarch 16:02, 24 Jan 2005 (UTC)

I don't think we've reached an impasse yet, but perhaps we (presently) disagree on what this article is about. Bayes' theorem is not about Bayesian-anything. It is a simple consequence of the definition of conditional probability. I don't think that this article should be about Bayesian decision theory, inference, probability or any other such approach to the analysis of uncertainty.

Even if my assertion that people "should come to appreciate that we cannot go around assigning probabilities to just anything" is misplaced (and I'm happy to agree that it is), the word 'event' is what probabilitists use to denote things to which we can assign probabilities. I cannote speak for Bayesian statisticians, as (despite doing my undergraduate degree in the field) I now do so little statistics that I can avoid declaring my allegiance. But, again, I don't believe that this article is about that at all. (I am aware of strong Bayesian constructions of probability theory, but they are not considered standard, by any means.)

What do you think of: "Bayes' theorem is a result in probability theory, which gives the conditional probability of A given B (where these are events, or simply things to which we may assign probabilities) in terms of the conditional probability of B given A and the (marginal) probabilities of A and B alone."

The main problem I have with the event business is that it's not necessary, and not helpful, in this context. Being told that A and B are elements of a sigma-algebra simply won't advance the understanding of the vast majority of readers -- this is the "not helpful" part. One can make a lot of progress in probability without introducing sigma-algebras until much later in the game -- this is the "not necessary" part. I'd prefer to say A and B are variables -- this avoids unnecessary assumptions. "A and B are simply things to which we may assign probabilities" is OK by me too. For what it's worth, Wile E. Heresiarch 16:24, 25 Jan 2005 (UTC)

The events article isn't that bad; the majority of it concerns a set of simple examples corresponding to the "things to which we may assign probabilities" definition. Of course, it also mentions the definition of events in the context of sigma algebras, but that is as it should be, too (after all, the term is in common use in that context). If you have qualms with the way the events article is presented, perhaps that needs attention, but I don't see that this should be a problem for Bayes' theorem. It seems a little POV to avoid use of the conventional term for "things to which we may assign probabilities" on the grounds that its formal definition, which does not appear in this article and is not the focus of the article on the term itself, may be difficult for some (even many) people to understand. Cheers, Ben Cairns 05:54, 26 Jan 2005 (UTC).

OK, so you saw the "not helpful" part. Can you address the "not necessary" part? Btw I don't have any desire or intent to change the event article. Wile E. Heresiarch 00:31, 27 Jan 2005 (UTC)

I think my comment above covers this to some exent, but to clarify... While the topic can certainly be explained without reference to events, we could just as easily discuss apes without calling them by that name—or worse, by calling them 'monkeys'—but that would obscure the facts that apes are (a) called 'apes', and (b) are not monkeys.

I have to say that I don't understand your resistance to using the word 'events', when you are satisfied with the (essentially) equivalent phrase, "things to which we may assign probabilities." How does adding the word detract from its elementary meaning? I don't deny that one can make a lot of progress without worrying about the details of constructing probability spaces, but providing a link which eventually leads to a discussion of those details hardly requires the reader to assimilate it all in one sitting.

Could you perhaps suggest, as a compromise, a way to present the material that (a) is clear even to the casual reader, and (b) at least hints that these things are called 'events'? Ben Cairns 04:25, 27 Jan 2005 (UTC).