Here's how I would explain the Monty Hall problem:

First, get the person to realize that switching doors will always flip your initial outcome. If you picked a losing door, switching gives you the winning one. If you picked the winning door, switching gives you a losing one. There is no winning-to-winning or losing-to-losing. If you switch, you will always get the opposite of whatever you originally picked.

Once this idea has been understood, ask them to consider the probability that your original choice was a losing door. Once they agree that it's 2/3 (2 losing doors out of 3 possible choices), you can reply "that's right, and if there's a 2/3 chance that your current choice is a losing door, then there's a 2/3 chance that you're in a situation where switching will give you the winning door. Therefore, the chance that switching will give you the winning door is 2/3!"

Bayes’ theorem says no such thing. Your friend is wrong, but please do not let this reflect badly on the Bayesian interpretation of probability in general—it has nothing to do with this.

And in fact a Bayesian approach, by being explicit about conditioning information, helps to show the “trick” behind the Monty Hall problem: when Hall opens a door, he reveals information about what’s behind the other two doors that the contestant didn’t have when he made his initial choice. The probabilities change in response to this new information.

I am not really sure what this has to do with bayesian vs. frequentists?

the bayesian theorem is a mathematical fact. It's true if we just accept some basic axioms. Bayesian/Frequentist divide is also pretty independent from the problem, since it's purely math-based? We are not estimating something...In fact, an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities.

>...an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities.

It seems simple, but the claim (not mine) was that the theorum states that since you are only choosing "once" you can't use an estimation, as that is based on collected data... which somehow doesn't exist or apply to you.

The reason is that somehow there are no odds/percentages/probabilities if there is only one instance of an event, or at least they don't change. (ie, 2 door, always 1 in 2 odds, period)

Normally I wouldn't detract from a thread like this, but participation seems low, so I can take the down votes in hopes of getting a response that helps me with this conundrum.

Bayesian statistics is a way of updating your beliefs about the value of some unknown number (or value) after doing an experiment. You have to be uncertain about some variable, and then you work backwards to figure out its value.

In the case of the Monty Hall problem, we have a complete understanding of what's going on, so there are no beliefs to update. There are no hidden rules or magic numbers we haven't been told about and we need to work out. You literally can't use Bayesian statistics if you understand a system completely.

Also, Bayesian statistics is a tool. It's not a law of nature. It can get the wrong answers. There is literally no guarantee that you will get the right answer when using it. Garbage In Garbage Out. It's just that there are cases where, empirically, it can be useful.

> we have a complete understanding of what's going on, so there are no beliefs to update.

This is not true, unless by 'complete understanding' you imply that everything must be deterministic and we understand every single variable.

However, probabilities can be interpreted as statements of lack of knowledge at certain points of a process, even if the entire process itself is understood completely.

For example, if I have a process where randomly half the time I get a 55% heads biased coin, and half the time I get a fair coin, then I flip it 10 times, I can absolutely apply Bayes' Theorem here to figure out probability I had the fair or the biased coin after seeing the result of the flips.

This is a system that I understand completely, and part of that system is a single unknown bit, which we're trying to put bounds on.

A correct Bayesian treatment of the Monty Hall problem is given in a cousin comment: https://news.ycombinator.com/item?id=17438740

No that's Bayes' theorem, not Bayesian inference. You haven't made any Bayesian assumptions.

Thanks for this explanation, it certainly expels some of my confusion.

It seems then based on your reply that my friend is possibly wrong for invoking Bayesian theory regarding the Monty Hall problem for the exact reason he claimed it was the only answer? (note, my friend is a brilliant person and dedicated engineer who understands math far above my level. I only program with statistics and the PhD's tell me what to do.)

In other words, his argument is incorrect in his assessment because we actually _do_ have previous knowledge? (He pointed out that Bayes theorum was used for analysing insurance premiums for new kinds of insurance because there was no previous data to work with, which fits with your statement.)

This seems like too easy an answer for him to have missed, which makes me suspect of my understanding of your reply.

(yes, my initial question was serious, I appreciate you replying)

Not only do we have previous knowledge, we know basically everything about what's going on. You only use Bayesian inference if there is something you don't know which can't be computed from the things you do know. Then, after doing an experiment, you work backwards to figure out the value of the things you didn't know.

Don't confuse Bayesian inference with Bayes' theorem, which is a theorem that doesn't have any philosophy attached to it. Bayesian inference makes use of this theorem on top of some philosophical assumptions, which may be questionable depending on how you use them.

Bayesian inference also requires you to give your "prior" beliefs, which are the beliefs you have before you do the experiment. If these beliefs are strongly biased away from the right answer, you will get Garbage Out.

Ok, that is really neat. I need to read more about this. I just did some searching on bayesian interference solution for the monty hall problem. And I am guessing they are applying the math whilst pretending to not have foreknowledge of the outcome?

https://sc5.io/posts/how-to-solve-the-monty-hall-problem-usi...

I certainly don't expect you to research this or explain every detail for me, I may have to take a class on this subject after I retire to satisfy my need to grasp this. Thank you for time, this was enlightening.

just to chime in. The monty-hall problem is a pure math (probability-theory) and not statistics-excersize, since we know all the probabilities beforehand and don't estimate.

It's very important to distinguish between bayesian inference and the bayesian theorem. The bayesian theorem can be proven and therefore must be true (if we accept the axioms). Bayesian inference has some real philosophical problems, since we often have no way to choose a justified prior. Frequentists approaches may also use the Bayesian theorem.

I always like the blog-posts of the angry statistician and he also has one about the monty-hall problem: http://angrystatistician.blogspot.com/2012/06/bayes-solution...

The important difference is that one is the probability of P(x == car), which is 1/3, and one is P(x == car | y == goat). Just apply the bayes theorem and calculate it yourself!

The easiest way to convince your friend of his error is to imagine the odds when there are n doors, and Monty opens n-2 of them, the ones which don't contain the prize and aren't your first pick.

How could Bayesian statistics be applied in this case? I’m wondering if the situation is just too “simple” to make it applicable.

Before Monty opens a door, so the prior probability, is 1/3 for each door.

After he opens a door, that door will have zero probability, as we know he opens a door without a car, then how to update the probabilities afterwards?

Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen. Chosen is 1/3 and not chosen is 2/3 before and after Monty opens a door, so perhaps there is no “Bayesian information” revealed by opening the door anyway.

> Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen.

Right. That might be the easiest way for this problem.

More straightforwardly, without that shortcut:

- Call the doors a, b, and c. Assume, without loss of generality, that we choose door a initially. Let the random variable X ∈ {a, b, c} be the door with the car.

- Our prior probability is uniformly distributed: Pr(X = a) = Pr(X = b) = Pr(X = c) = 1/3.

- The data Y that we collect is our observation of which door gets opened by the host. The likelihood function Pr(Y = y | X = x) is the probability of the observation being y (i.e., that Y = y), given that the underlying state is x (i.e., that X = x). The only non-zero likelihoods are Pr(Y = b | X = a) = Pr(Y = c | X = a) = 1/2 and Pr(Y = c | X = b) = Pr(Y = b | X = c) = 1.

- Bayes' theorem, Pr(X = x | Y = y) = Pr(Y = y | X = x)·Pr(X = x)/Pr(Y = y), gives the answer, the posterior probability, which should be seen as a function of x. The denominator Pr(Y = y) = ∑ Pr(Y = y | X = x), sum over x ∈ {a, b, c}, is a normalising factor that makes the posterior probability distribution sum to 1. It is also the probability we, at the start of the game, assign to Y = y. In our problem, Pr(Y = a) = 0 and Pr(Y = b) = Pr(Y = c) = 1/2.

How about you put in the numbers and see if it comes out right or if I have made a mistake? :-)

Edit: Sorry, I just realised that we could have made it simpler by assuming that the host opens, say, door b.

Brilliant. Yes, the wrong solutions are highlighted by the door opening event, and thus your odds of success increase when you eliminate bad outcomes.

Judea Pearl uses the Monty Hall problem in his latest book, and when he draws out the causal network, the issue becomes obvious. The game-show host chooses which door to open based on two factors: he has to choose a door you didn't, and he has to choose a door with a goat.

Hence, the door he chooses to open is probabilistically informative about your initial choice. Treating the door he opened as information on which to condition does change the posterior.

He’s wrong, you can show switching doors improves your performance with Bayes’ theorem.

Edit: I removed my entire comment, it seems that there is a lot more online now about Bayesian theory than when I had looked this up, or my google-fu was off when I was researching this before.

Thanks for giving me new search terms to consider. I have more arguments and examples to read through now. :P

“Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, ‘Do you want to pick door No. 2?’ Is it to your advantage to switch your choice?”

Let A be the event that the car is behind door 1, B that the car is behind door 2, and C that the car is behind door 3. Let E be the event that the host opens door 3. We assume that the car is initially equally likely to be behind each door and that the host opens a door with a goat at random, never opening the door we picked.

Since A, B, and C are exhaustive and mutually exhaustive propositions, we can calculate the marginal probability of E by using the law of total probability:

P(E) = P(E ∧ A) + P(E ∧ B) + P(E ∧ C).

Bayesians like to define joint probability from conditional probability instead of the reverse; that is, define P(A ∧ B) as P(A | B) P(B) instead of P(A | B) as P(A ∧ B) / P(B).

So P(E ∧ A) = P(E | A) P(A). P(E | A) is 1/2 because we picked door 1, the car is behind door 1, and the host chooses at random a door that has a goat, of which there are two: 2 and 3. P(A) is 1/3. Therefore P(E ∧ A) is 1/2 × 1/3 = 1/6.

Similarly, P(E ∧ B) = P(E | B) P(B). P(E | B) is 1 because we picked door 1 so the host will not open door 1 and we assume the car is behind door 2 so the host will not open door 2, leaving only door 3 to be opened. P(B) is 1/3. Therefore P(E ∧ B) is 1 × 1/3 = 1/3.

P(E ∧ C) = P(E | C) P(C). P(E | C) is 0 because the host will never open the door the car is behind. P(C) is 1/3. Therefore P(E ∧ C) is 0 × 1/3 = 0.

So P(E) = 1/6 + 1/3 + 0 = 1/2. We know that the host opened door 3 (this is E), so the car cannot be behind door 3. How likely is it to be behind door 1? By Bayes’ theorem,

P(A | E) = P(E | A) P(A) / P(E).

We said earlier that P(E | A) is 1/2, P(A) is 1/3, and P(E) is 1/2. So P(A | E) = (1/2 × 1/3) / (1/2) = 1/3.

Given E, the car must be behind door 1 or door 2 since the host opened door 3. Therefore the sum of P(A | E) and P(B | E) must be 1. P(A | E) is 1/3, so P(B | E) is 2/3. The car is more likely to be behind door 2 than door 1. We initially picked door 1, so, if we want the car, we should switch.

Thanks, this looks quite a bit more succinct than other examples of the math. I think I will have to make an app that renders out results using this. :)

> "frequentist" as if it were some kind of pejorative.

Ah, so he was right about that, at least! :-)

Does this comment deserve a downvote? I think he's asking a genuine question.