Data Mining Lessons from "Moneyball"

If you're a data miner, there's a good chance you saw the movie, Moneyball. For me, the movie was entertaining, but the book was really enlightening. I found it to be a source of numerous and familiar lessons about data mining - while couched in the intriguing context of major league baseball. In this article, I'd like to share some of these lessons with you.

Lessons 1, 2 & 3: Ask Questions, Pay Attention to Mistakes, and Don't Be Defensive

Early in my career as a data miner, I was conditioned to believe that my role was to answer questions. Someone would hand me a data set and say, "Tell me who our best customers are", or "Find out why fewer customers are renewing." I would trundle off to my cube and dig into the data to try to provide an answer. Have you had a similar experience?

As I worked on more and more projects, however, I found myself turning around to those clients and colleagues to ask questions - about the meaning of the data, about the population underlying the data, about the time frame of interest, about the outcome of interest, about the measures of success, etc. And I asked myself, "Am I asking too many questions?"

Luckily, I came across an article [f1] by Kevin Kelly that made me feel good about asking so many questions. He wrote:

"Some day answers (correct answers!) will be so cheap that the really valuable things will be questions. A really good question will be worth a thousand correct answers."

Read more.