KDnuggets Home » News » 2010 » Mar » Poll: concise laws frequent in large data  ( < Prev | 10:n05 | Next > )

Poll Results: Are there concise laws in large data?


 
  
35% think concise laws are frequent, but concise laws are probabilistic due to noise randomness inherent in human or biological systems


The previous KDnuggets Poll asked:
Do you think that there are concise, mathematical laws (patterns) in large datasets in business, social, and biology data?

(Here is a relevant discussion about concise laws in physics and "Unreasonable Effectiveness of Data")

About 35% of KDnuggets readers felt that such laws are frequent, while 36% thought that such laws are present only occasionally.

Gaussian curve Most comments were about inherent noise present in human or biological systems, but noise does not prevent us from finding statistical patterns - e.g. the Gaussian curve is a beautiful example of a concise law that expresses a fundamental property of normal distribution, albeit that property is only valid in a "statistical" sense.


Ed R. thought that

you may be able to identify some simple, generalizable relationships ... Just because you can fit an equation to data does *not* mean that the equation has any deep connection to the underlying system!

Tom Dietterich wrote

The whole power of data mining methods is that they scale with the complexity of the data and therefore they can detect very complex patterns. We don't really need them if we are looking for simple patterns.
Much of the human social and information world is characterized by "arbitrary complexity". Fred Brooks has defined computer science as the study of arbitrary complexity, and data mining is a key tool for doing this.

Ross Bettinger noted

I doubt that concise formulations of human behavior analogous to physics, e.g., F=ma, will be frequently found among living organisms and populations because individual variation often contributes as much noise as signal to any detection algorithm

Here are full results and comments of KDnuggets Poll:
Concise, mathematical laws (patterns) in large datasets


KDnuggets Home » News » 2010 » Mar » Poll: concise laws frequent in large data  ( < Prev | 10:n05 | Next > )