Features

Rexer Analytics is pleased to share the results of our 2nd Annual Data Miner Survey. We also want to extend our deepest thanks to all KDnuggets readers who participated in the survey.

2008 HIGHLIGHTS:

• 34-item survey of data miners, conducted on-line in early 2008
• 348 responses from individuals in 44 countries
• The most commonly used algorithms are decision trees, regression, and cluster analysis. The use of time series and survival analysis increased this year.
• Dirty data, data access issues, and explaining data mining to others remain the top challenges faced by data miners
• Data miners are most likely to use descriptive stats, outlier detection, and face validity to identify / address dirty data
• Data miners spend only 20% of their time on actual modelling. More than a third of their time is spent accessing and preparing data.
• Data mining is playing an important role in organizations. Half of data miners indicate their results are helping to drive strategic decisions and operational processes.
• The most prevalent concerns about data mining utilization are: resistance to using data mining in contexts where it would be beneficial, insufficient training of some data miners, and lack of model refreshing
• SPSS Clementine was identified as the primary software used by more data miners than any other software product. SPSS and SAS continue to dominate the software market. However, Statistica, R, and the Salford products saw increased usage this year.
• In selecting their analytic software, data miners place a high value on dependability, the ability to handle very large datasets, and quality output
• The findings vary somewhat depending on the domain in which the data miner works, the tools used, geography, and several other dimensions

The Third Annual Data Miner Survey will be launched in January 2009. Please send us your comments on the 2008 survey and ideas for the 2009 survey. Many people provided input that helped shape the 2008 survey, and we'd like to incorporate your ideas into the 2009 data miner survey.

Thanks

-- Karl and the Rexer Analytics research team