Publications

Posted by randfish on Sun (9/9/07) at 08:25 PM Analytics

Way back in November of last year, I suggested a linkbait idea to the wide community on a comparison of the major analytics vendors. Despite an incredible amount of work and effort required to put it together, Eric Enge of Stone Temple Consulting prevailed and produced the incredibly valuable, must read research report - The 2007 Web Analytics Shootout.

...

Rand: The analytics comparison project took you more than 9 months to complete - that's an incredible amount of work. Can you tell us some of the things that made it so challenging and time consuming to create and run?

Eric Enge: Part of it was that it simply took about 3 months to get vendors and web sites on board. With each vendor, we had to learn the installation procedures, and then we had to work through them with each participating site.

Once the code was setup on all the participating sites, the actual data collection part proceeded without a ton of effort by us. We spent many months collecting data, so that we could look for any interesting patterns that might emerge.

However, we were also interested in putting together a qualitative analysis. To be able to write that up, we had to spend many hours in each application trying to do different things with them, to find out what we could, and couldn't do. We also spent many hours on the phone with the various vendors having them educate us about that.

Last, but not least, analyzing the data took quite a bit of effort. We collected massive reams of numbers, and I had to do a lot of analysis before meaningful information started to emerge.

...

Read more.