Software

GPeerReview (http://code.google.com/p/gpeerreview/) is a tool for making digital-signatures. It is specifically designed for use with peer reviews. For example, a conference might use it to sign the papers it accepts. If an author were to display such a signature on his resume, potential employers could easily verify the signature without having to search for the paper on the conference's website. Perhaps more significantly, individual researchers can also use this tool to review and sign each other's papers. This may provide somewhat of an alternative to traditional venues for peer reviews.

(here's where the data-mining comes in...)

As researchers review and sign each others' papers, a decentralized web of reviewers and papers is naturally formed. (The tool implicitly formalizes this social network by including a hyperlink to the reviewer's publication list in each review that he makes.) The structure of this web (or social network) will naturally mirror the structure of the research community itself. In theory, therefore, it could be mined to reveal information about the influence or well-connectedness of individual researchers with the research community. (This differs from mining citation graphs because reviews have significantly different meaning than citations, and because each review contains several scores which give meta-meaning to the link.)

In the past, people have tried to solve perceived problems in scientific peer review by combining services like arXiv.org with Slashdot-like moderation systems. Such systems are always doomed to failure because they rely on a "magic formula" to rank every author. Even if everyone could agree on the formula, the system would be gamed into oblivion. If, however, data-mining is the solution to automated evaluation, then everyone can emphasize their own preferred qualities, and there is no single formula to game.