In Bayesian probability theory, a marginal likelihood function is a likelihood function integrated over some variables, typically model parameters. Bayesian probability interprets the concept of Probability as 'a measure of a state of knowledge'. Probability theory is the branch of Mathematics concerned with analysis of random phenomena In Statistics, the likelihood function (often simply the likelihood) is a function of the Parameters of a Statistical model that plays a key role Integrated likelihood is a synonym for marginal likelihood. Evidence is also sometimes used as a synonym, but this usage is somewhat idiosyncratic. "Marginal likelihood" is the most commonly-used of these three terms.

For any likelihood function of two or more variables, marginal likelihoods with respect to any subset of the variables can be defined. Let a denote the subset of variables marginalized (i. e. , integrated). Let b denote the other variables. Let x denote observed data. Given the likelihood function p(x|a, b), the marginal likelihood of b is

where p(a|b) is the distribution of a conditional on b. The marginal likelihood of a is computed in an analogous way, by exchanging the roles of a and b.

In a widely-used application, the marginalized variables are parameters for a particular type of model, and the remaining variable is the identity of the model itself. In this case, the marginalized likelihood is the probability of the data given the model type, not assuming any particular model parameters. Writing θ for the model parameters, the marginal likelihood for the model M is

This quantity is important because the posterior odds ratio for a model M₁ against another model M₂ involves a ratio of marginal likelihoods, the so-called Bayes factor:

which can be stated schematically as

The Bayes factor is an object of central importance in Bayesian model comparison. In Statistics, the use of Bayes factors is a Bayesian alternative to classical Hypothesis testing. A common problem in Statistical inference is to use data to decide between two or more competing models

Unfortunately, marginal likelihoods are generally difficult to compute. Exact solutions are known for a small class of distributions. In general, some kind of numerical integration method is needed, either a general method such as Gaussian integration or a Monte Carlo method, or a method specialized to statistical problems such as the Laplace approximation or Gibbs sampling. In Numerical analysis, numerical integration constitutes a broad family of algorithms for calculating the numerical value of a definite Integral, and by extension In Numerical analysis, a quadrature rule is an approximation of the definite integral of a function, usually stated as a Weighted sum of function Monte Carlo methods are a class of Computational Algorithms that rely on repeated Random sampling to compute their results For the optimization method called "steepest descent" see Gradient descent. In Mathematics and Physics, Gibbs sampling is an Algorithm to generate a sequence of samples from the joint probability distribution of two or

See also

• Bayes factor

• Bayesian model comparison

References

• Charles S. In Statistics, the use of Bayes factors is a Bayesian alternative to classical Hypothesis testing. A common problem in Statistical inference is to use data to decide between two or more competing models Bos. "A comparison of marginal likelihood computation methods". In W. Härdle and B. Ronz, editors, COMPSTAT 2002: Proceedings in Computational Statistics, pp. 111-117. 2002. (Available as a preprint on the web: [1])
• The on-line textbook: Information Theory, Inference, and Learning Algorithms, by David J.C. MacKay.