Publications

Bloor Research, Philip Howard, 6 August, 2009

IBM's recent announcement that it is to acquire SPSS not only has implications in terms of the completeness of IBM's analytics offering but it also has ramifications for the future of data mining as a whole. It is the latter upon which I will focus.

There are two aspects to this: the first is with regard to the choices available to users wishing to use data mining tools and the second is with respect to in-database data mining. I will discuss this second issue first.

Traditionally, in order to perform data mining operations you extracted the data from the data warehouse or mart, into the mining tool where you processed it. The problem with this is that there is a significant performance hit from the extraction of all that data. Of course, you could reduce that impact by sampling the data, but then you lose accuracy. As a result, the trend recently has been to perform mining processes "in-database". And one of the first announcements we can expect after the acquisition is complete is that IBM will be implementing SPSS functionality in DB2.

Now, both Oracle and Microsoft have the ability to process mining algorithms in-database (as indeed does IBM) but neither of these has a tool like SPSS, so this will give IBM a significant advantage over these two. However, almost the entirety of the rest of the warehousing market has no such capabilities, the exceptions being where SAS has been working with Teradata, Netezza and others to implement SAS capabilities in-database. Also notable (and to be commended for their foresight) is Netezza, which acquired NuTech Solutions last year, though this is more of a tool for building predictive applications than a data mining tool per se.

Read more.