Bayesian Updating: From Prior and Likelihood to Posterior

Bayesian updating revises an uncertain judgment when new evidence arrives. After observing evidence $E$, the probability of hypothesis $H$ is:

$$P(H\mid E)=\frac{P(E\mid H)P(H)}{P(E)}$$

This event-probability form requires $P(E)>0$ and follows from the two factorizations of $P(H\cap E)$. For a hypothesis with zero prior mass, multiplying by a finite likelihood cannot give it positive posterior mass. Continuous observations are handled with likelihood densities rather than probabilities of exact points; see the NIST treatment of Bayes' formula.

The four parts of the formula

• $P(H)$ is the prior probability, the assessment of the hypothesis before the current evidence is observed.
• $P(E\mid H)$ is the likelihood, the probability of seeing the evidence if the hypothesis is true. It answers "How common would this evidence be under the hypothesis?" It is different from $P(H\mid E)$.
• $P(E)$ is the probability of the evidence. If the hypotheses $H_i$ are mutually exclusive and exhaustive, then $P(E)=\sum_i P(E\mid H_i)P(H_i)$. This term normalizes the results so that the posterior probabilities sum to 1.
• $P(H\mid E)$ is the posterior probability, the assessment after incorporating the evidence.

When comparing hypotheses, the same relationship can be written as:

$$P(H\mid E)\propto P(H)P(E\mid H)$$

Compatibility alone does not guarantee an increase. For $0<P(H)<1$, the posterior exceeds the prior exactly when $P(E\mid H)>P(E\mid\neg H)$. If both likelihoods are 0.8, the evidence fits both explanations and leaves the prior unchanged. The likelihood ratio determines the direction of the update; the prior also affects its final size.

Base rates: the 0.2% and 99% example

Suppose a condition affects $0.2\%$ of a population. A test has $99\%$ sensitivity: it returns a positive result for $99\%$ of people who have the condition. It also has $99\%$ specificity: it returns a negative result for $99\%$ of people who do not have it. Let $H$ mean that the person has the condition and $+$ mean a positive test result. For a person selected randomly from this population, Bayes' formula gives:

$$P(H\mid +)= \frac{0.99\times0.002} {0.99\times0.002+0.01\times0.998} \approx16.6\%$$

Among 10,000 people, 20 are expected to have the condition and 19.8 of them to test positive. Of the remaining 9,980 people, 99.8 are expected to receive a false positive. True positives therefore make up about $16.6\%$ of all positive results.

Sensitivity and specificity are not the same as the probability of having a condition after a positive result. That probability also depends on prevalence, the tested population, and the testing conditions. This is a mathematical example, not a claim about a particular disease or product, and it cannot replace medical assessment.

Sequential updating

One posterior can become the prior for the next update:

$$P(H\mid E_1,E_2) \propto P(H\mid E_1)P(E_2\mid H,E_1)$$

Failing to find a target in a searched area is also evidence. If detection would usually succeed when the target is present, an unsuccessful search lowers the posterior probability that the target is there. The update depends on whether the search area was complete, how reliable detection was, and whether missed detections were modeled correctly.

LK-99 offers a compact example. Early observations raised the plausibility of a room-temperature, ambient-pressure superconductivity explanation. Later replications, resistance measurements, and material analyses lowered it. The material analysis concerned a Cu₂S impurity, not iron sulfide. The example shows beliefs changing over several rounds of evidence without rewriting the initial uncertainty after the fact.

Dependent evidence must not be counted twice

The joint likelihood of two pieces of evidence is generally:

$$P(E_1,E_2\mid H) = P(E_1\mid H)P(E_2\mid E_1,H)$$

The second factor reduces to $P(E_2\mid H)$ only when the evidence is conditionally independent given $H$. Reports derived from one original source, indicators affected by the same measurement error, or observations produced by one latent cause are dependent. Treating such redundant evidence as independent can count the same information more than once and produce excessive confidence. For dependence in general, the direction of the error depends on the joint likelihoods under the competing hypotheses.

Naive Bayes classifiers assume conditional independence between features to simplify computation. This is a modeling approximation, not a claim that words in natural language are genuinely independent.

What the model can establish

A Bayesian model makes its priors, candidate hypotheses, observation errors, and evidence dependencies explicit. A missing hypothesis, a poorly chosen prior, a mistaken likelihood, or ignored dependence can still produce a misleading posterior.

Bayesian models can fit some patterns of perception and judgment. Such a fit does not prove that the brain performs exact Bayesian calculations. Bayesian methods also appear in some machine-learning systems, but that does not reduce every form of AI to one inference procedure. Bayesian analysis is a tool for studying cognition and AI, not a proven unified theory of every mind or intelligent system.

Relation to macro scenarios

This page explains the general probability structure. A Bayesian Method for Macro Scenarios applies it to mutually exclusive scenarios, likelihood-ratio judgments, resolvable forecasts, and calibration scores. The macro guide focuses on recording and testing forecasts instead of repeating these definitions.

Sources

• NIST: Bayes' formula, priors, likelihoods, and posteriors
• CDC: prevalence, sensitivity, specificity, and positive predictive value
• Nature: LK-99 replication results and the Cu₂S analysis
• Stanford: naive Bayes text classification
• Original Bilibili video: 这个简单的公式，蕴含着直觉和智能的密码

My reading · Opened notes are remembered in this browser.