Datafiniti offers Web Data via SQL

ProgrammableWeb, Garrett Wilkin, October 11th, 2011

If there's a new dawn in the age of search, Datafiniti is the sun. They are the first to take a new and profound approach to search in years. Most search engines return a list of links to web pages, but Datafiniti has much bigger plans. Instead of links, they return a set of data. A search for a burrito in Texas would give you a list of restaurants complete with goodies like reviews in addition to the important bits like name and address. The dataset itself has been out there for years, but its the aggregation and presentation that's different. With the Datafiniti API, it is now possible for developers to easily integrate that data into web applications.

This is incredibly profound because the task of normalizing and serving data sets is a big one. For any given subject area, there are multiple APIs to learn and master in order to collect just the most relevant information. This is a real problem for programmers and Datafiniti is making an earnest effort tp solve it. They provide one simple API call that consists of a token and an SQL query. The results can be provided in JSON or CSV. This makes it possible for programmers to use Datafiniti as a great simplifying normalization point, but it also makes the data accessible to the moderately tech savvy spreadsheet user. Datafiniti will serve copy and pastable datasets that will likely find their way into slide show presentations. There's a page dedicated toward helping people up the syntax learning curve and a screencast to walk you through some typical queries.

An important aspect of this service is that its built on a powerful web crawling architecture developed by 80legs which is now a part of Datafiniti. In a recent conversation CEO Shion Deysarkar described the crawling system as "completely transient and ephemeral cloud computing grid" which allows Datafiniti to operate nimbly and cheaply. It also gives their service a true competitive advantage. They've got a serious engine under the hood that should keep the data set healthy on all metrics of coverage, recency, and accuracy. A high level of quality is Datafiniti's stategy for adoption.

Read more.

See also Datafiniti builds a webscale search engine for data, GigaOM, Oct 10, 2011.

Gregory Piatetsky-Shapiro: my query for getting all restaurants in Brookline, MA returned 111 items which I could purchase for a total of $0.86 - a very reasonable deal

Comments:

GregoryPS
Datafiniti data schema is limited, but once you understand what is available, it is very simple to use. Have patience if your query is not instant