Cox's theorem

Summary

Cox's theorem, named after the physicist Richard Threlkeld Cox, is a derivation of the laws of probability theory from a certain set of postulates.[1][2] This derivation justifies the so-called "logical" interpretation of probability, as the laws of probability derived by Cox's theorem are applicable to any proposition. Logical (also known as objective Bayesian) probability is a type of Bayesian probability. Other forms of Bayesianism, such as the subjective interpretation, are given other justifications.

Cox's assumptions edit

Cox wanted his system to satisfy the following conditions:

  1. Divisibility and comparability – The plausibility of a proposition is a real number and is dependent on information we have related to the proposition.
  2. Common sense – Plausibilities should vary sensibly with the assessment of plausibilities in the model.
  3. Consistency – If the plausibility of a proposition can be derived in many ways, all the results must be equal.

The postulates as stated here are taken from Arnborg and Sjödin.[3][4][5] "Common sense" includes consistency with Aristotelian logic in the sense that logically equivalent propositions shall have the same plausibility.

The postulates as originally stated by Cox were not mathematically rigorous (although more so than the informal description above), as noted by Halpern.[6][7] However it appears to be possible to augment them with various mathematical assumptions made either implicitly or explicitly by Cox to produce a valid proof.

Cox's notation:

The plausibility of a proposition   given some related information   is denoted by  .

Cox's postulates and functional equations are:

  • The plausibility of the conjunction   of two propositions  ,  , given some related information  , is determined by the plausibility of   given   and that of   given  .
In form of a functional equation
 
Because of the associative nature of the conjunction in propositional logic, the consistency with logic gives a functional equation saying that the function   is an associative binary operation.
  • Additionally, Cox postulates the function   to be monotonic.
All strictly increasing associative binary operations on the real numbers are isomorphic to multiplication of numbers in a subinterval of [0, +∞], which means that there is a monotonic function   mapping plausibilities to [0, +∞] such that
 
  • In case   given   is certain, we have   and   due to the requirement of consistency. The general equation then leads to
 
This shall hold for any proposition  , which leads to
 
  • In case   given   is impossible, we have   and   due to the requirement of consistency. The general equation (with the A and B factors switched) then leads to
 
This shall hold for any proposition  , which, without loss of generality, leads to a solution
 
Due to the requirement of monotonicity, this means that   maps plausibilities to interval [0, 1].
  • The plausibility of a proposition determines the plausibility of the proposition's negation.
This postulates the existence of a function   such that
 
Because "a double negative is an affirmative", consistency with logic gives a functional equation
 
saying that the function   is an involution, i.e., it is its own inverse.
  • Furthermore, Cox postulates the function   to be monotonic.
The above functional equations and consistency with logic imply that
 
Since   is logically equivalent to  , we also get
 
If, in particular,  , then also   and   and we get
 
and
 
Abbreviating   and   we get the functional equation
 

Implications of Cox's postulates edit

The laws of probability derivable from these postulates are the following.[8] Let   be the plausibility of the proposition   given   satisfying Cox's postulates. Then there is a function   mapping plausibilities to interval [0,1] and a positive number   such that

  1. Certainty is represented by  
  2.  
  3.  

It is important to note that the postulates imply only these general properties. We may recover the usual laws of probability by setting a new function, conventionally denoted   or  , equal to  . Then we obtain the laws of probability in a more familiar form:

  1. Certain truth is represented by  , and certain falsehood by  
  2.  
  3.  

Rule 2 is a rule for negation, and rule 3 is a rule for conjunction. Given that any proposition containing conjunction, disjunction, and negation can be equivalently rephrased using conjunction and negation alone (the conjunctive normal form), we can now handle any compound proposition.

The laws thus derived yield finite additivity of probability, but not countable additivity. The measure-theoretic formulation of Kolmogorov assumes that a probability measure is countably additive. This slightly stronger condition is necessary for the proof of certain theorems.[citation needed]

Interpretation and further discussion edit

Cox's theorem has come to be used as one of the justifications for the use of Bayesian probability theory. For example, in Jaynes it is discussed in detail in chapters 1 and 2 and is a cornerstone for the rest of the book.[8] Probability is interpreted as a formal system of logic, the natural extension of Aristotelian logic (in which every statement is either true or false) into the realm of reasoning in the presence of uncertainty.

It has been debated to what degree the theorem excludes alternative models for reasoning about uncertainty. For example, if certain "unintuitive" mathematical assumptions were dropped then alternatives could be devised, e.g., an example provided by Halpern.[6] However Arnborg and Sjödin[3][4][5] suggest additional "common sense" postulates, which would allow the assumptions to be relaxed in some cases while still ruling out the Halpern example. Other approaches were devised by Hardy[9] or Dupré and Tipler.[10]

The original formulation of Cox's theorem is in Cox (1946), which is extended with additional results and more discussion in Cox (1961). Jaynes[8] cites Abel[11] for the first known use of the associativity functional equation. János Aczél[12] provides a long proof of the "associativity equation" (pages 256-267). Jaynes[8]: 27  reproduces the shorter proof by Cox in which differentiability is assumed. A guide to Cox's theorem by Van Horn aims at comprehensively introducing the reader to all these references.[13]

Baoding Liu, the founder of uncertainty theory, criticizes Cox's theorem for presuming that the truth value of conjunction   is a twice differentiable function   of truth values of the two propositions   and  , i.e.,  , which excludes uncertainty theory's "uncertain measure" from its start, because the function  ,[a] used in uncertainty theory, is not differentiable with respect to   and  .[14] According to Liu, "there does not exist any evidence that the truth value of conjunction is completely determined by the truth values of individual propositions, let alone a twice differentiable function."[14]

See also edit

Notes edit

  1. ^ Liu uses the symbol ∧ as the "minimum operator", most likely referring to a binary operation that takes two numbers and returns the smaller (or minimum) of the two.

References edit

  1. ^ Cox, R. T. (1946). "Probability, Frequency and Reasonable Expectation". American Journal of Physics. 14 (1): 1–10. Bibcode:1946AmJPh..14....1C. doi:10.1119/1.1990764.
  2. ^ Cox, R. T. (1961). The Algebra of Probable Inference. Baltimore, MD: Johns Hopkins University Press.
  3. ^ a b Stefan Arnborg and Gunnar Sjödin, On the foundations of Bayesianism, Preprint: Nada, KTH (1999) — ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/06arnborg.ps[permanent dead link]ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/06arnborg.pdf[permanent dead link]
  4. ^ a b Stefan Arnborg and Gunnar Sjödin, A note on the foundations of Bayesianism, Preprint: Nada, KTH (2000a) — ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/fobshle.ps[permanent dead link]ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/fobshle.pdf[permanent dead link]
  5. ^ a b Stefan Arnborg and Gunnar Sjödin, "Bayes rules in finite models," in European Conference on Artificial Intelligence, Berlin, (2000b) — ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/fobc1.ps[permanent dead link]ftp://ftp.nada.kth.se/pub/documents/Theory/Stefan-Arnborg/fobc1.pdf[permanent dead link]
  6. ^ a b Joseph Y. Halpern, "A counterexample to theorems of Cox and Fine," Journal of AI research, 10, 67–85 (1999) — http://www.jair.org/media/536/live-536-2054-jair.ps.Z Archived 2015-11-25 at the Wayback Machine
  7. ^ Joseph Y. Halpern, "Technical Addendum, Cox's theorem Revisited," Journal of AI research, 11, 429–435 (1999) — http://www.jair.org/media/644/live-644-1840-jair.ps.Z Archived 2015-11-25 at the Wayback Machine
  8. ^ a b c d Edwin Thompson Jaynes, Probability Theory: The Logic of Science, Cambridge University Press (2003). — preprint version (1996) at "Archived copy". Archived from the original on 2016-01-19. Retrieved 2016-01-19.{{cite web}}: CS1 maint: archived copy as title (link); Chapters 1 to 3 of published version at http://bayes.wustl.edu/etj/prob/book.pdf
  9. ^ Michael Hardy, "Scaled Boolean algebras", Advances in Applied Mathematics, August 2002, pages 243–292 (or preprint); Hardy has said, "I assert there that I think Cox's assumptions are too strong, although I don't really say why. I do say what I would replace them with." (The quote is from a Wikipedia discussion page, not from the article.)
  10. ^ Dupré, Maurice J. & Tipler, Frank J. (2009). "New Axioms for Rigorous Bayesian Probability", Bayesian Analysis, 4(3): 599-606.
  11. ^ Niels Henrik Abel "Untersuchung der Functionen zweier unabhängig veränderlichen Gröszen x und y, wie f(x, y), welche die Eigenschaft haben, dasz f[z, f(x,y)] eine symmetrische Function von z, x und y ist.", Jour. Reine u. angew. Math. (Crelle's Jour.), 1, 11–15, (1826).
  12. ^ János Aczél, Lectures on Functional Equations and their Applications, Academic Press, New York, (1966).
  13. ^ Van Horn, K. S. (2003). "Constructing a logic of plausible inference: A guide to Cox's theorem". International Journal of Approximate Reasoning. 34: 3–24. doi:10.1016/S0888-613X(03)00051-3.
  14. ^ a b Liu, Baoding (2015). Uncertainty Theory. Springer Uncertainty Research (4th ed. 2015 ed.). Berlin, Heidelberg: Springer Berlin Heidelberg : Imprint: Springer. pp. 459–460. ISBN 978-3-662-44354-5.

Further reading edit

  • Fine, Terrence L. (1973). Theories of Probability : An examination of foundations. New York: Academic Press. ISBN 0-12-256450-2.
  • Smith, C. Ray; Erickson, Gary (1989). "From Rationality and Consistency to Bayesian Probability". In Skilling, John (ed.). Maximum Entropy and Bayesian Methods. Dordrecht: Kluwer. pp. 29–44. doi:10.1007/978-94-015-7860-8_2. ISBN 0-7923-0224-9.