Features

I was unaware of the Information Awareness Office (IAO) until this issue of KDNuggets [been busy with regular work].

I liked the coin example to illustrate the efficacy of mining over many variables (a.k.a. "multi-coin toss") to get the 19 terrorists out of 300,000,000 Americans. [A lot of computation there!]

I certainly agree that data mining can be used to narrow down the search and the more variables, the smaller the final candidate group can be.

My concern is how these systems will be used and what conclusions will be drawn from them. If these systems become very capable, there may be a tendency to trust their judgement allowing some automated decision making. Most KDD systems are based on probability based tests of significance. When there are only 19 terrorists out of 300,000,000 Americans, how adequate would the traditional 0.01->0.05 significance levels be? There will always need to be final deterministic work done to check whether a recommended candidate actually is a terrorist or whether there is still a terrorist out there that has been missed, i.e. we never want to punish the innocent and let the guilty go free.

The world has changed quite a bit since 9/11 and whilst our data is more private between citizens it seems it is going to be much less private between the citizen and the Government. I hope the Government powers never fall into the wrong hands and that initiatives such as the IAO will lead to a safer world.