JMP is celebrating the International Year of Statistics by honoring an influential statistician each month. This month we take a look at Thomas Bayes, a minister and mathematician whose name is literally attached to statistical inference. Very few details are known about Bayes, but his impact on statistics and science in general is remarkable considering that he published only two papers in his lifetime.
His primary contribution was Bayes Rule, a law of conditional probability that bears his name. Bayes Rule led to the field of Bayesian Inference, a very powerful approach to data analysis that continues to gain momentum. Now there is an International Society for Bayesian Analysis devoted to promoting and developing Bayesian methods. We don’t have much to say about the life of Thomas Bayes, but we can focus on his impact on the field of statistics.
Thomas Bayes is thought to have been born in Hertfordshire, England, sometime between July 1701 and April 1702. His family had made their fortune in the cutlery industry, and his father, Joseph Bayes, was a nonconformist Presbyterian minister. Thomas followed his father’s advice to become a minister, entering the University of Edinburgh in 1719 to study divinity and prepare for the ministry. While at Edinburgh, he developed an interest in mathematics. In particular, he was interested in infinite series and approximations. Bayes died in 1761 at the age of 59, possibly from rheumatism.
Bayes published only two works, the first theological and the second mathematical in nature. In Divine Benevolence, or an Attempt to Prove That the Principal End of the Divine Providence and Government is the Happiness of His Creatures (1731), Bayes argued that goodness or benevolence motivated God’s actions in the world. In 1736, he anonymously published An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of the Analyst. Bayes was a loyal supporter of Sir Isaac Newton, and this paper defended the foundation of Newton’s calculus against criticisms presented in a book called The Analyst. Although he had only a single mathematical publication to his name, Bayes was a very well-respected mathematician. He was elected to the Royal Society in 1742 for his work on calculus, unpublished papers on geometry, and his service as a referee or critic of mathematical research.
So how was Thomas Bayes able to have so much influence on the field of statistics with only a single paper (on calculus) to his credit? After his death, his friend Richard Price went through a collection of notebooks where Bayes had kept notes and manuscripts in progress. In one of the notebooks, Price found an essay where Bayes was attempting to solve an inverse probability problem. For the binomial distribution with probability of success p, Bayes set out to find the distribution of p given an observed number of successes. He found that if he assumed that p was uniformly distributed, then the distribution of p given the observed number of successes has a beta distribution. Price read this essay to the Royal Society in 1763, two years after Bayes had passed away. Price later went on to use this result to try to prove the existence of God.
The binomial problem detailed in Bayes’ notebook was a special case of what has now become known as Bayes Theorem. Bayes Theorem (also known as Bayes Law or Bayes Rule) can be stated
where Pr(A) is the probability of event A happening and Pr(A | B) is the probability of event A happening given that event B has happened. Bayes Theorem is thought of as inverting or turning around conditional probabilities.
Let’s think about a simple example. Suppose that I tell you that my only sibling drives a sports car. I don’t mention whether I have a brother or a sister, so you might be interested in the probability that I have a brother. We assume that 15% of males drive sports cars (completely made up), 10% of women drive sports cars (also completely made up), and that the population is approximately half male and half female (made up, but more believable). Then the probability that my sibling is male given that he or she drives a sports car is
So by having prior knowledge about tendencies to drive sports cars, you are able to infer that there is a 60% chance that I have a brother.
Bayes Rule serves as the foundation and motivation for the extremely popular field of Bayesian data analysis. The key idea behind Bayesian analysis is that we can use prior information in conjunction with our observed data to make better decisions. Bayesian ideas have been used to approach problems in virtually every corner of statistics. In fact, there is a long list of techniques that contain the words Bayes or Bayesian. From that list, the phrase Bayesian poisoning caught my eye. Luckily though, Bayesian poisoning is not a movement by frequentists to cut down on the number of Bayesian statisticians. Instead, it is a technique for email spammers to beat spam filters that rely on Bayesian methods for detecting offending messages. Receiving spam certainly is annoying, but at least Bayesian poisoning isn’t as sinister as the name may suggest.
While we may not know much about the life of Thomas Bayes, one big question about him stands out from the rest: Was he the first person to discover Bayes Theorem? Laplace independently rediscovered and expanded Bayes Theorem in the early 1800s, but there is also evidence that Bayes Theorem was discovered by someone else several years before Bayes. Stigler (1983) suggests that Nicholas Saunderson, a math professor at Cambridge, may have been the first person to derive Bayes Theorem. In fact, Stigler even uses Bayes Rule to attempt to suggest that Saunderson is more likely than Bayes to have been first to make the discovery! The mystery surrounding the first person to discover Bayes Theorem makes me appreciate that I have all the conveniences of the Internet at my disposal. I’m only a few clicks of the mouse away from having a copy of just about any research paper that I want. I can’t begin to imagine how much more difficult it would have been to keep up with research in the 1700s.
So let’s not forget to celebrate Thomas Bayes during this International Year of Statistics. His simply stated law of conditional probabilities has led to an incredibly powerful and popular branch of statistics. Perhaps it is no coincidence that 2013 is also the 250th anniversary of Richard Price presenting Bayes work to the Royal Society.
Bellhouse, D.R. (2004) "The Reverend Thomas Bayes, FRS: A Biography to Celebrate the Tercentenary of His Birth", Statistical Science, 19, 3-43.
Stigler, S. M. (1983) "Who Discovered Bayes's Rule?" The American Statistician, 37, 290-296.