Briefs

CNet (09/29/04); Kanellos, Michael

The extension of search technology beyond text is illustrated by Marvel, a system from IBM designed to comb through thousands of hours of video and audio to retrieve specific clips. The ability of existing search engines to retrieve video clips or images is still text-based, and limited to a small portion of manually labeled files. Manual labeling is a laborious, time-consuming job that is becoming even more difficult with the rapid growth of information in need of classification.

John R. Smith of IBM Research says the goal of the Marvel project is "to index content without using text or manual annotations." Marvel automatically categorizes and retrieves clips by assigning modifiers such as "outdoor," "indoor," "cityscape," or "engine noise" to the action depicted in the clip; the system employs a form of artificial intelligence called support vector machines, in which a computer is taught to tag pieces of data with the equivalent of a yes or no value. The amount of data condensed in even short video clips is enormous: The Marvel research team has recognized 166 distinctive dimensions for color-based queries, and the system will scan both the audio and visual tracks to counterbalance the many other dimensions that have to be ignored in order for Marvel to function efficiently. The project is a collaborative venture between IBM Research, libraries, and news organizations such as CNN; the first Marvel prototype, which was spotlighted in August at a Cambridge University conference, is capable of searching a database of over 200 hours of broadcast news video using 100 different descriptive labels, which IBM hopes to increase to 1,000 by April 2005.

Here are some IBM examples of Marvel Queries.

Here is the rest of the story.