The hidden key beyond just causation vs correlation

See what others that are more successful than you are doing, and do that! – Random Guru

There’s pieces of truth to this statement, but the problem behind simply modeling is not being able to see the entire process.

Imagine that everyone that is a professional actor has had 10 movies before they have their break out performance. The guru advice would be to simply get 10 movies under your belt as quickly as possible. But the reality is that making 10 movies isn’t what you need to model, it’s the process of acting in a movie, and then internally, the actor studies how to make their next movie better by increasing their skills.

To a modeler, the answer is to just mimic the results of “make 10 movies,” but the reality was it took 10 movies of practice, and then study behind the scenes, to get good.

There’s no such thing as modeling someone 100%, and there will always be parts of the puzzle that are unseen.

One of the best ways to determine if a model is missing critical data is to see if anyone else has the same output, but different results. If so, you’re missing something.

But there’s a bigger problem than just that…

If you notice someone with the same output but different results you MAY not be missing any critical information, but dealing with a larger problem which is probabilities.

If you’re modeling a gambler on how to get rich, just because someone bought a ticket and didn’t win as opposed to the other person that bought a ticket and won, doesn’t mean that you’re missing some part of the formula to model. It means you’re dealing with probabilities.

When was the last time you modeled someone and got different results, and worse, didn’t know if you were just missing some part of the formula, or you were dealing with issues of probability?