Briefs

By Elise Ackerman, Mercury News, 11/05/2008

Behind Yahoo's push to open up Web search and advertising is software powerful enough to sort through the entire Library of Congress in less than half a minute.

The software, called Hadoop, is part of Yahoo's massive computing grid and is transforming the way that Yahoo and corporate giants like IBM extract meaning from enormous streams of data. Universities are also using the code - an open-source version of software Google relies on for daily operation - to train a new generation of computer scientists and engineers.

"It makes it possible to actually take advantage of all the computers that we have hooked together," said Larry Heck, vice president of search and advertising sciences at Yahoo.

Hadoop improves the relevance of ads Yahoo shows on the Internet by analyzing the company's endless flow of data - now well over 10 terabytes a day - on the fly. As users click from Yahoo Mail to Yahoo Search to Yahoo Finance and back again, Hadoop helps figure out what ad, if any, is likely to catch someone's attention.

The key lies in mining insights from mind-boggling amounts of data. If a woman repeatedly reads reviews of sport-utility vehicles, then clicks on automotive classifieds and then orders a book about helping a child adjust to kindergarten, she might be in the market for a new family-size car, according to a Yahoo sales presentation.

As part of the push for more openness, Yahoo will be using the technology not Advertisement only to boost ad sales on its own Web sites, but on sites owned by the 796 members of a newspaper consortium that is working with the search giant to sell more advertising at better prices. The Mercury News and its parent company, MediaNews, are members of the partnership.

Read more.