Prev
INTRODUCTION
11Ants Analytics Customer Churn Analyzer beat 85% of submissions in the world's most prestigious international data mining contest - with only 55 minutes of human work involved. This white paper has been written to enable the experiment to be repeated by others, and to enable companies evaluating various predictive analytics solutions to benchmark performance of other solutions both in terms of time taken and efficacy of models.
To our knowledge there is no other solution that can build superior models to 11Ants Customer Churn Analyzer with so little human effort required. This applies whether you are an experienced data scientist or a relative novice to the discipline and offers material time and quality advantages to both constituencies
WHY THIS IS IMPORTANT
Too often automated model building solutions are dismissed as unsuitable for experts, or compromised. While
this may have been true of previous generation technologies, 11Ants predictive analytics technologies are
capable of routinely beating humans at complex predictive analytics problems, and - equally importantly -
with a fraction of the time and energy expended. This white paper has been prepared to demonstrate the
speed and efficacy of models on a tangible real world problem.
THE COMPETITION
KDD Cup is the annual Data Mining and Knowledge Discovery competition organized by
SIGKDD - ACM Special Interest Group on Knowledge Discovery and Data Mining,
the leading professional organization of data miners.
The KDD Cup 2009
focused on customer relationship predictions. The European telecommunications ORANGE
provided customer data for analysis. The data set consisted of 50,000 customer records with 216 variables per
customer. The tasks were to build three separate propensity models for each customer record as outlined
below in the task description. The models were:
1) Churn Propensity
2) Appetency Propensity
3) Up-sell Propensity
More details can be found at www.kddcup-orange.com/evaluation.php
THE PROCESS
A detailed description of the process for those wishing to repeat the experiment can be found in Appendix A.
This provides an overview of the process. Entire time committed from start to finish 35 minutes.
1. Download data from contest site.
2. Open data in Microsoft Excel.
3. Split data into analysis set and test set.
4. Press 'Analyze Data' button.
5. Leave running for a few hours.
6. Score completion supplied (blind) test set.
7. Upload results back to competition site for ranking.
THE RESULTS
The models generated by 11Ants Customer Churn Analyzer out-performed models built by 85% of the competition submissions. This was after 40 hours of processing time. A snapshot of the general performance of the models at hours 3, 5, 9, 15, 24 and 40 are shown below. You will note that even at hour three the model outperformed 78% of the submissions. It should be noted that though computer processing time was up to 40 hours, actual human time did not exceed 30 minutes total.
Thw full white paper can be downloaded from the right hand side of this page:
www.11antsanalytics.com/products/11AntsCustomerChurnAnalyzer/