Publications

Today, let me try and convince you to avoid the accuracy metric in favor of other metrics such as precision and recall.

Accuracy is often the starting point for analyzing the quality of a predictive model. Accuracy is also probably the first term that comes to mind when non-experts think about how to evaluate the quality of a prediction. As shown below, accuracy measures the ratio of correct predictions over the total number of cases evaluated.

What about the business relevance of accuracy? Surprisingly, this is a difficult question. It seems obvious that the ratio of correct predictions over all cases should be a key metric for determining the business impact of a predictive model. Yet, the value of the accuracy metric is dubious. In fact, it is often trivially easy to create a predictive model with high accuracy, and such trivial models can be useless despite of high accuracy. Similarly, when comparing the business impact of two alternative predictive models, it may well be the less accurate model that is more beneficial to the user organization.

See the rest at www.bloglines.com/blog/Tilmann?id=13