Software

Bixo is an open source web mining toolkit that runs as a series of Cascading pipes on top of Hadoop.

By building a customized Cascading pipe assembly, you can quickly create specialized web mining applications that are optimized for a particular use case.

There's also Bixolabs, which is an elastic web mining platform built w/Bixo, Cascading & Hadoop for Amazon's cloud (EC2).

It makes it easy to quickly build efficient & cost effective web mining apps that run in Amazon's Elastic Compute Cloud.

With Bixolabs you can focus on what you know best � using the data � without the challenges of building out a reliable, scalable web crawling and data processing workflow.