Publications

Karl J. Brazier, Analysis & Insight Consultant, UK, writes

A brief comment on item 25 in KDNuggets 2007 no. 10, by Innar:

I have encountered exactly this attitude when doing DM work in an application with both social policy and econometric components. While it didn't prevent us doing the work because our socio-economic collaborator was an open-minded person who could see the potential we were offering, it led to most of our data mining work being expunged from a paper in order to get published in an economic journal (Fiscal Studies).

I'd speculate that the problem may stem from, apart from an inevitable defensiveness in the face of what for these applications is still seen as novelty, a couple of things:

1. A lack of appreciation of the size of hypothesis space that needs to be searched in order to avoid being misled by preconceptions, in relation to the capacity of human beings to do this without computational assistance and

2. A failure to discriminate between the form of a model, as constrained by its so-called "language bias", which defines the hypothesis space to be searched, and the particular parameterisations of the model to be searched within that space. I think DM approaches should be sellable to the more open-minded economists by emphasising the role of their expert knowledge in defining the former. It may end up with rather more constrained search spaces than in other fields, but that's not necessarily such a bad thing.

How many open-minded economists are out there, I wouldn't like to guess. Maybe we just got very lucky with our collaborator. But if, so, then I guess it's always possible to by-pass the closed-minded and go straight to the politicians and business persons who take the decisions.

Geert Bilcke, Dataminer, Belgium, writes

Performing a single optimization for two different things at the same time can be interesting, but is not optimal. As women (or females of other species) sometimes optimize their male partners separately for two different purposes (one as biological father of their children, the other to provide the means to care for them) you can optimize a model separately for the two purposes : explicability or predictability. For direct marketing purposes I prefer a good black box model over a moderate white box model.

What's this?