Bayesian Marbles

Suppose there are three marbles, each either black or white with presumed equal probability, in a covered container. Based on this premise, the probability of all three marbles being white is 1/8.  Now suppose we blindly reach in and pull out a marble - it is white. We return that marble to the container and make another blind selection - it again is white. We return the marble and make another pick – white again. So, we have made three random selections and come out with three white marbles. What, now, is the probability that all three marbles are white?

Bayes Theorem is the easiest way of computing the answer. For any two events X and Y, the probability of both X and Y being true is clearly equal to the probability of {X is true} times the probability of {Y is true given that X is true}. This can be written as  Pr{X and Y} = Pr{X} Pr{Y|X},  where the symbol "|" is read as "given". From this it follows that

So if we define Y as the event {All three marbles are white} and X as the event {All three of our selections are white}, then obviously the probability that all 3 marbles are white and all 3 selections are white is just equal to the probability that all 3 marbles are white, which is 1/8. So we only need to determine Pr{X}, the probability that all 3 selections are white. 

As already noted, the unconditional probability that all 3 marbles are white is 1/8, and we are certain to have all 3 selections white in that case. The unconditional probability that exactly 2 marbles are white is 3/8, in which case we have (2/3)³ probability of all 3 selections being white. The unconditional probability of exactly 1 white marble is 3/8, in which case we have (1/3)³ probability of all 3 selections being white. Therefore, the overall probability of all 3 selections being white is

So, we've determined that, of all possible outcomes, 1/4 result in all 3 selections being white, and 1/8 have all 3 marbles white and all 3 selections white. It follows that of the cases when all 3 selections are white, exactly half are cases with all 3 marbles being white. Hence the conditional probability of all three marbles being white, given that all three of our selections was white, is 1/2.

Now, sometimes students think that the probability of all the marbles being white should be 2/9, based on the notion that it equals the probability of selecting a different marble on each of our three trials, which equals (1)(2/3)(1/3)= 2/9. It's true that if we select a different marble each time, and if all 3 selections are white, then all three marbles are white, but the converse is not true. If we do not select a different marble each time, it's still possible that all 3 marbles are white. So, to work the problem this way, we need to consider all the possibilities: We have 2/9 chance of selecting 3 different marbles, and the probability of all 3 selections being white is 1/8.  We have 6/9 chance of selecting one marble twice and another marble once, and in this case our chances of all 3 selections being white are 1/4. We have 1/9 chance of selecting the same marble all three times, and in this case our chance is 1/2 of all three selections being white. Combining these, the total probability of all 3 selections being white is

so again the probability of all three marbles being white, given that all 3 selections are white, is (1/8)/(1/4) = 1/2.

For a related problem, suppose a box contains six white marbles and an unknown number of blue marbles, but not more than six. Three marbles are removed successively without replacement, and all are blue. If a 4th marble is removed, what is the probability that it is white? This problem, again, calls for some Bayesian analysis, to infer the probability of a prior condition from some subsequent results. In this case we are trying to figure out the probabilities that the original number of blue marbles was 0,1,2,..,or 6, given the fact that we drew 3 blue marbles in a row. Again we use a form of Bayes theorem

which is really just a consequence of the intuitive fact that Pr{A and B} = Pr{A}Pr{B|A}, and by symmetry we also have Pr{A and B} = Pr{B}Pr{A|B}, so we can equate these two to give the above version of Bayes' theorem.

The probability that there were originally 5 blue marbles, given that we drew 3 blue marbles in a row, is equal to [the probability that we would draw 3 in a row given that there were originally 5 blue marbles] times [the probability that there were originally 5 blue marbles] divided by [the probability that we would draw 3 in a row].

Notice that if we assume the prior probability of 5 blue marbles originally is the same as 4 or 3 or 2 or 1 or 6, then that factor is the same for all the possibilities. Also, the overall probability of drawing 3 blue marbles (unconditionally) is the same for all cases. Thus, the probabilities are proportional to the respective values of Pr{B|A}, i.e., the probability that we would draw 3 blue in a row given that there were originally k blue marbles (k = 3, 4, 5, 6). The results are

Letting P_t = p3 + p4 + p5 + p6, the overall result is found by multiplying the probability of drawing a white on the next marble given each of these four possible priors, so we have

which is about 74.5%. 

Incidentally, suppose we re-worded the problem like this: “A box contains from 9 to 12 marbles, 6 of which are white, and the remainder red. If someone randomly draws four marbles (without replacement) and the first three of those four are red, what is the probability that the fourth one is white?” The answer is the same as with the original wording. We can make the original problem explicitly symmetrical over all seven of the equally likely prior conditions (which consist of there being 0, 1, 2,..., or 6 blue marbles at the start) by letting p_k denote the probability that we would draw 3 blue marbles in a row given that there were originally k blue marbles. Then define

to give the "weights" (p_k/p_t) of the seven possible conditions. Of course, it turns out that p₀ = p₁ = p₂ = 0, so the solution reduces to the same form as given originally. In other words, the problem has the same answer with the original wording as with the alternate wording. The only assumption that's needed (in either case) is that the prior probabilities of having 3, 4, 5, or 6 blue marbles are all equal. It doesn't matter if these priors are each 1/4 (meaning there are only four possible priors) or each 1/7 (meaning there could have been 0, 1, or 2 blue marbles also). The reason is that the subsequent condition of drawing 3 blue marbles in a row automatically sets the "post" probabilities of the 0,1,2 cases to zero. The priors of 1/7 or 1/4 are just used to determine the proportionality between the resulting non-zero weights. As long as their priors are all the same (all 1/4 or all 1/7) the final answer will be the same.

Return to MathPages Main Menu