Need to crawl billions of Web pages? There's an app for that (KDnuggets News 09:18, item 38, Briefs)

KDnuggets : News : 2009 : n18 : item38

Briefs

Need to crawl billions of Web pages? There's an app for that

ComputerWorld, Eric Lai, 22.09.2009 | IDG News Service

80legs Inc. has officially launched its service, which brings supercomputer-scale data mining of the Web to companies, and even individuals.

The Houston, Texas-based startup leverages a grid of 50,000 servers to search and crunch millions of Web pages within minutes, CEO Shion Deysarkar told Computerworld on Monday ahead of the DEMOfall 09 conference in San Diego.