CHAPTER 1 Bayess TheoremConditional probabilityThe fundamental idea behind all Bayesian statistics is Bayess theorem, which is sur prisingly easy to derive, provided that you understand conditional probability. So well start with probability, then conditional probability, then Bayess theorem, and on to Bayesian statistics.A probability is a number between 0 and 1 (including both) that represents a degree of belief in a fact or prediction. The value 1 represents certainty that a fact is true, or that a prediction will come true. The value 0 represents certainty that the fact is false.Intermediate values represent degrees of certainty. The value 0.5, often written as 50%, means that a predicted outcome is as likely to happen as not. For example, the probability that a tossed coin lands face up is very close to 50%.A conditional probability is a probability based on some background information. For example, I want to know the probability that I will have a heart attack in the next year. According to the CDC, “Every year about 785,000 Americans have a first coronary attack (http:/www.cdc.gov/heartdisease/facts.htm).”The U.S. population is about 311 million, so the probability that a randomly chosen American will have a heart attack in the next year is roughly 0.3%.But I am not a randomly chosen American. Epidemiologists have identified many fac tors that affect the risk of heart attacks; depending on those factors, my risk might be higher or lower than average.I am male, 45 years old, and I have borderline high cholesterol. Those factors increase my chances. However, I have low blood pressure and I dont smoke, and those factors decrease my chances.1Plugging everything into the online calculator at http:/hp2010.nhlbihin.net/atpiii/calcu lator.asp, I find that my risk of a heart attack in the next year is about 0.2%, less than the national average. That value is a conditional probability, because it is based on a number of factors that make up my “condition.”The usual notation for conditional probability is p A B , which is the probability of A given that B is true. In this example, A represents the prediction that I will have a heart attack in the next year, and B is the set of conditions I listed.Conjoint probabilityConjoint probability is a fancy way to say the probability that two things are true. I write p A and B to mean the probability that A and B are both true.If you learned about probability in the context of coin tosses and dice, you might have learned the following formula:p A and B =p A p BWARNING: not always trueFor example, if I toss two coins, and A means the first coin lands face up, and B means the second coin lands face up, then p A =p B =0.5, and sure enough, p A and B =p A p B =0.25.But this formula only works because in this case A and B are independent; that is, knowing the outcome of the first event does not change the probability of the second. Or, more formally, p B A = p B .Here is a different example where the events are not independent. Suppose that A means that it rains today and B means that it rains tomorrow. If I know that it rained today, it is more likely that it will rain tomorrow, so p B A p B .In general, the probability of a conjunction isp A and B =p A p B Afor any A and B. So if the chance of rain on any given day is 0.5, the chance of rain on two consecutive days is not 0.25, but probably a bit higher.2 | Chapter 1: Bayess Theorem1. Based on an example from http:/en.wikipedia.org/wiki/Bayes_theorem that is no longer there.The cookie problemWell get to Bayess theorem soon, but I want to motivate it with an example called the cookie problem.1 Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla cookies and 10 chocolate cookies. Bowl 2 contains 20 of each.Now suppose you choose one of the bowls at random and, without looking, select a cookie at random. The cookie is vanilla. What is the probability that it came from Bowl 1?This is a conditional probability; we want p Bowl 1 vanilla , but it is not obvious how to compute it. If I asked a different questionthe probability of a vanilla cookie given Bowl 1it would be easy:p vanilla Bowl 1 =3/4Sadly, p A B is not the same as p B A , but there is a way to get from one to the other: Bayess theorem.Bayess theoremAt this point we have everything we need to derive Bayess theorem. Well start with the observation that conjunction is commutative; that isp A and B =p B and Afor any events A and B.Next, we write the probability of a conjunction:p A and B =p A p B ASince we have not said anything about what A and B mean, they are interchangeable. Interchanging them yieldsp B and A =p B p A BThats all we need. Pulling those pieces together, we getp B p A B =p A p B AThe cookie problem | 3Which means there are two ways to compute the conjunction. If you have p A , you multiply by the conditional probability p B A . Or you can do it the other way around; if you know p B , you multiply by p A B . Either wa