Bayes Theorem

Bayes' Theorem is a theorem in probability theory named for Thomas Bayes (1702–1761).

It is used for updating probabilities by finding conditional probabilities given new data. This simplest case involves a situation in which probabilities have been assigned to each of several mutually exclusive alternatives H₁, ..., H_n, at least one of which may be true. New data D is observed. The conditional probability of D given each of the alternative hypotheses H₁, ..., H_n is known. What is needed is the conditional probability of each hypothesis H_i given D. Bayes' Theorem says

$P(H_{i}\mid D)={\frac {P(H_{i})P(D\mid H_{i})}{P(H_{1})P(D\mid H_{1})+\cdots +P(H_{n})P(D\mid H_{n})}}.$

The use of Bayes' Theorem is sometimes described as follows. Start with the vector of "prior probabilities", i.e. the probabilities of the several hypotheses before the new data is observed:

$P(H_{1}),\dots ,P(H_{n}).\,$

Multiply these term-by-term by the "likelihood vector":

$P(D\mid H_{1}),\dots ,P(D\mid H_{n}),\,$

getting

$P(H_{1})P(D\mid H_{1}),\dots ,P(H_{n})P(D\mid H_{n}).\,$

The sum of these numbers is not (usually) 1. Multiply all of them by the "normalizing constant"

$c=(P(H_{1})P(D\mid H_{1})+\cdots +P(H_{n})P(D\mid H_{n}))^{-1},\,$

getting

$cP(H_{1})P(D\mid H_{1}),\dots ,cP(H_{n})P(D\mid H_{n}).\,$

The result is the "posterior probabilities", i.e. conditional probabilities given the new data:

$P(H_{1}\mid D),\dots ,P(H_{n}\mid D).\,$

In epidemiology, it is used to obtain the probability of disease in a group of people with some characteristic on the basis of the overall rate of that disease and of the likelihoods of that characteristic in healthy and diseased individuals. In clinical decision analysis it is used for estimating the probability of a particular diagnosis given the appearance of some symptoms or test result.