Features

Q8. GPS: You also have a responsibility for Yahoo Research Labs -- can you tell us about some of the more exciting projects there?

Usama Fayyad: At Yahoo! Research, we have been working on some really interesting projects around prediction markets and Search technology. These are typically featured on next.yahoo.com Here are a couple of examples:

Mindset
is a very innovative tool that analyses the content of web pages and constructs a score of "degree of commercialness" of a page. We then utilize this automatic categorization (which incidentally utilizes the world's fastest SVM [Support Vector Machine] learning algorithm -- also invented and built at Yahoo! Research), to allow the user to set a slider bar to reflect his/her intent.

If you are making a query and you are shopping for something, then commercial content in the search results is what you want. On the other hand, if you are researching something, you want to filter out commercial pages in favor of educational content pages. So moving the slider to the other end will re-order the search results, and many relevant pages that appear as rank 100 or 200 start appearing in the top 10. People who use this slider quickly get addicted to it (I am).

I encourage your readers to check it out. Note that this is only one dimension, and one can imagine many more that are important and many more sliders can be exposed: family-friendliness, personal, etc... Ultimately, if we know you enough, we can set these multitudes of sliders automatically for you.

Tech Buzz Game
is another example of an innovative and deep concept we launched. Back in March 2005, we launched an innovative idea that is now becoming fashionable: a futures market in concepts! We had been working on that idea for over a year. The idea was to pick an area, in this case technology buzz as measured by search keyword activity, and provide a stock market for Tech Buzz Keywords.

We launched this as a game, but in the background we were doing the science: building the theory of what is happening and evaluating the predictive value of this marketplace in forecasting new trends in the technology market. The experiment was a huge success with over 20K players participating in the game we launched.

The winner actually used the optimal strategy derived by David Pennock, one of our Research Scientists who recently won the MIT Technology Review 35 Under 35 Award for this work. In fact, the winner beat randomized robots proving that the strategy works. This is an example of how we aim to build the sciences underpinning the Internet and Interactive Media.

GPS: the conclusion of the interview will be published in the next issue of KDnuggets News.