There’s a famous math problem called the Monty Hall Effect, or Monty Hall Problem. Basically it deals with probability and how you should act in light of new information. Monty Hall, a famous game show host, would present three doors. Behind one was a car or some other grand prize, typically the other two doors are hiding a goat when the problem is discussed. Whether or not Monty did that in reality, I have no idea.
So, you’ve got three doors and the car is behind one. Monty asks you to choose one, he then reveals where one of the goats is and asks you if you want to switch your choice. Here’s where the problem comes in, because most people see two doors left, figure it’s a 50/50 chance of getting it right, and so stick with their choice. That’s the wrong move, you should always switch. Information is why you should switch.
The crux of the problem is that probabilities apply to events, not states of being. So the car is behind one door definitely. When you choose initially you have a 1 in 3 chance of being right and a 2 in 3 chance of being wrong. Now, the thing is when Monty reveals the goat behind one of the other two doors it doesn’t change the odds of the car being behind either remaining door, it just gives you more information to make a choice. There was still, at the time of choosing, a 2 out of three chance you’d be wrong. Those odds still apply.
One way to look at that illustrates it better is to assume 100 doors, and you have to choose one. So you do, then Monty throws open 98 of those doors to show there’s no prize, just a goat, so the car is behind one of the remaining two doors. Do you really think your odds at the initial choice were 50/50? They weren’t, and since it’s not a new decision you’re making, because technically it’s the same choice and you just have more information, the chances are still 98 to 1 that you’re wrong, and that applies to both remaining doors. However, whether dealing with three doors or 100, one other way to see how this applies is to view it as two choices, your door and all the others bundled together. Point being, it’s still more likely than not that your initial choice was wrong, and switching is better.
Now it may look like a 50/50 chance, but recall even though you have the chance to switch your choice, you are not changing the state of being of the result; the car is still behind the door it was always behind. Now, if Monty said he’d reveal where the goats are and leave you two choices, but then switch which one the car was behind, then you’re dealing with a 50/50 chance. But, when the car doesn’t move, the odds stay the same.
This is a useful, if not directly applicable scenario, to illustrate one of the major mistakes people make in hiring, specifically when they look to establish predictive measures. The common approach is to measure some aspect of the people who are performing well right now and apply that to the hiring process. There are various tests that do this, but the underlying problem is that what you know after you’ve hired them isn’t necessarily going to predict success when it’s used to inform a choice on who to hire now, because it’s not necessarily what you based your initial decision on.
A simple example is if you find everyone who is successful in a particular position is good at math, you then look for people who are good at math. However, did you know that before hiring them all? Were they good when they started, or did they get good at the particular math required? Testing successful employees isn’t useless, but you have to keep in mind that it’s eagle-eye hindsight. Who did you reject that would have done just as well if not better?
The real approach would be a pre and post assessment of hiring. How did you rate your choice of hire, and what was it based on, and what did you know about there person, before you ascertained whether or not their performance was up to par? Measuring some aspect of them after you already know they’re successful and looking for that in candidates doesn’t quite get it right. You should always do pre hire and post hire assessments, and then look for what was truly predictive of a good hire, not for what a good hire does after you already know they’re good. The problem is you could be using the wrong information, and using irrelevant assessment criteria for rating potential hires.
While the parallel with Monty Hall’s problem isn’t spot on, it does deal with the use of information in making decisions, and making the wrong judgement about how new information actually affects your choices. What matters in the end is this: what information can you reliably gather about a candidate before you hire them, and what information in that set has the highest predictive value?