## Bayesian Balls

```Suppose there are three balls, each either black or white with
presumed equal probability, in a covered container.  Based on this
premise, the probability of all three balls being white is 1/8.  Now
suppose we blindly reach in and pull out a ball - it is white.  We
return that ball to the container and make another blind selection -
it again is white.  We return the ball and make another pick - white
again.  So, we have made three random selections and come out with
three white balls.  What, now, is the probability that all three balls
are white?

Bayes Theorem is the easiest way of computing the answer.  For any
two events X and Y, the probability of both X and Y being true is
clearly equal to the probability of {X is true} times the probability
of {Y is true given that X is true}.  This can be written as
Pr{X and Y} = Pr{X} Pr{Y|X},  where the symbol "|" is read as "given".
From this it follows that

Pr{X and Y}
Pr{Y|X} = -------------                 (1)
Pr{X}

So if we define Y as the event {All three balls are white} and X as
the event {All three of our selections are white}, then obviously the
probability that all 3 balls are white AND all 3 selections are white
is just equal to the probability that all 3 balls are white, which is
1/8.  So we only need to determine Pr{X}, the probability that all
3 selections are white.

As already noted, the unconditional probability that all 3 balls are
white is 1/8, and we are certain to have all 3 selections white in
that case.  The unconditional probability that exactly 2 balls are
white is 3/8, in which case we have (2/3)^3 probability of all 3
selections being white.  The unconditional probability of exactly
1 white ball is 3/8, in which case we have (1/3)^3 probability of
all 3 selections being white.  Therefore, the overall probability
of all 3 selections being white is

(1/8)*(1) + (3/8)*(2/3)^3 + (3/8)*(1/3)^3  =  1/4

So, we've determined that, of all possible outcomes, 1/4 result in
all 3 selections being white, and 1/8 have all 3 balls white AND
all 3 selections white.  It follows that of the cases when all 3
selections are white, exactly half are cases with all 3 balls being
white.  Hence the conditional probability of all three balls being
white, given that all three of our selections was white, is 1/2.

Now, sometimes students think that the probability of all the balls
being white should be 2/9, based on the notion that it equals the
probability of selecting a different ball on each of our three trials,
which equals (1)(2/3)(1/3)= 2/9.  It's true that if we select a
different ball each time, and if all 3 selections are white, then
all three balls are white, but the converse is not true.  If we do
NOT select a different ball each time, it's still possible that all
3 balls are white.  So, to work the problem this way, we need to
consider all the possibilities:  We have 2/9 chance of selecting 3
different balls, and the probability of all 3 selections being white
is 1/8.  We have 6/9 chance of selecting one ball twice and another
ball once, and in this case our chances of all 3 selections being
white are 1/4.  We have 1/9 chance of selecting the same ball all
three times, and in this case our chance is 1/2 of all three
selections being white.  Combining these, the total probability of
all 3 selections being white is

(2/9)(1/8) + (6/9)(1/4) + (1/9)(1/2)  =  1/4

so again the probability of all three balls being white, given that
all 3 selections are white, is (1/8)/(1/4) = 1/2.

For a related problem, suppose a box contains six white balls and an
unknown number of blue balls, but not more than six.  Three balls are
removed successively without replacement, and all are blue. If a 4th
ball is removed, what is the probability that it is white?  This
problem, again, calls for some Bayesian analysis, to infer the
probability of a prior condition from some subsequent results.  In
this case we are trying to figure out the probabilities that the
original number of blue balls was 0,1,2,..,or 6, given the fact that
we drew 3 blue balls in a row.  Again we use a form of Bayes theorem

Pr{A}
Pr{A|B} = Pr{B|A} -----
Pr{B}

which is really just a consequence of the intuitive fact that
Pr{A and B} = Pr{A}Pr{B|A}, and by symmetry we also have
Pr{A and B} = Pr{B}Pr{A|B}, so we can equate these two to give the
above version of Bayes' theorem.

The probability that there were originally 5 blue balls, given that
we drew 3 blue balls in a row, is equal to [the probability that we
would draw 3 in a row GIVEN that there were originally 5 blue balls]
times [the probability that there were originally 5 blue balls]
divided by [the probability that we would draw 3 in a row].

Notice that if we assume the "a priori" probability of 5 blue balls
originally is the same as 4 or 3 or 2 or 1 or 6, then that factor is
the same for all the possibilities.  Also, the overall probability
of drawing 3 blue balls (unconditionally) is the same for all cases.
Thus, the probabilities are PROPORTIONAL to the respective values of
Pr{B|A}, i.e., the probability that we would draw 3 blue in a row
GIVEN that there were originally k blue balls (k=3,4,5,6).  The
results are

3 2 1           4 3 2          5  4 3           6  5 4
p3  = - - -     p4 = -- - -    p5 = -- -- -     p6 = -- -- --
9 8 7          10 9 8         11 10 9          12 11 10

Letting Pt = p3 + p4 + p5 + p6, the overall result is found by
multiplying the probability of drawing a white on the next ball given
each of these four possible priors, so we have

p3         p4           p5           p6           677
-- (1)  +  -- (6/7)  +  -- (6/8)  +  -- (6/9)  =  ---
pt         pt           pt           pt           909

which is about 74.5%.

Incidentally, suppose we re-worded the problem like this:

A box contains from 9 to 12 balls, 6 of which are white,
and the remainder red.  If someone randomly draws four
balls (without replacement) and the first three of those
four are red, what is the probability that the fourth
one is white?

The answer is the same as with the original wording.  We can make
the original problem explicitly symmetrical over all seven of the
equally likely prior conditions (which consist of there being 0,
1, 2,..., or 6 blue balls at the start) by letting pk denote the
probability that we would draw 3 blue balls in a row GIVEN that
there were originally k blue balls.  Then define

pt = p0+p1+p2+p3+p4+p5+p6

to give the "weights" (pk/pt) of the seven possible conditions.  Of
course, it turns out that p0 = p1 = p2 = 0, so the solution reduces
to the same form as given originally.  In other words, the problem
has the same answer with the original wording as with the alternate
wording.  The only assumption that's needed (in either case) is that
the prior probabilities of having 3, 4, 5, or 6 blue balls are all
equal.  It doesn't matter if these priors are each 1/4 (meaning they
are are only four possible priors) or each 1/7 (meaning there could
have been 0, 1, or 2 blue balls also).

The reason is that the subsequent condition of drawing 3 blue balls
in a row automatically sets the "post" probabilities of the 0,1,2 cases
to zero.  The priors of 1/7 or 1/4 are just used to determine the
PROPORTIONALITY between the resulting non-zero weights.  As long as
their priors are all the same (all 1/4 or all 1/7) the final answer
will be the same.
```