KDD-CUP-98 Results

From: Ismail Parsa, iparsa@epsilon.com
Date: Wed, 26 Aug 1998 14:36:06 -0400
Subject: KDD-CUP-98 Individual Performance Summary

On behalf of the KDD-CUP-98 Committee, I am pleased to announce the winners of this year's KDD-CUP.
  1. The GOLD MINER award goes to:
    Urban Science Applications, Inc. with their software GainSmarts.
    (See Urban Science Story)

  2. The SILVER MINER award goes to:
    SAS Institute, Inc. with their software Enterprise Miner.

  3. The BRONZE MINER award goes to:
    Quadstone Limited with their software Decisionhouse.
    (See Quadstone Story)

The awards were be presented at KDD-98 in New York, NY on August 29, 1998.

KDD-CUP-98 PROGRAM COMMITTEE

Summary of Evaluation Results

KDD-CUP Committee evaluated the results based on the net revenue generated on the hold-out or validation sample.

The measure we used is:

Sum (the actual donation amount - $0.68) over all records for which the expected revenue (or predicted value of the donation) is over $0.68.

This measure is simple, objective and a direct measure of profit. Table 2 depicts the results. The participants are listed based on the last column. 

===============================================================
TABLE 1: Summary Statistics for Key Fields included in
         Validation Dataset
===============================================================
Field           N       MIN     MEAN    STD     MAX     SUM
===============================================================
Response        96,367  0       5.1%    21.9%   1       4,873
Donation Amount 96,367  $0      $0.79   $4.73   $500.00 $76,090
Profit Damt-.68 96,367  -$0.68  $0.11   $4.73   $499.32 $10,560
===============================================================


TABLE 2: KDD-CUP-98 Summary of Evaluation Results: Total Profits
         for Records with Predicted Donation > $0.68
===============================================================
Participant     N*      MIN     MEAN    STD     MAX     SUM**
===============================================================
GainSmarts      56,330  -$0.68  $0.26   $5.57   $499.32 $14,712
SAS             55,838  -$0.68  $0.26   $5.64   $499.32 $14,662
Quadstone       57,836  -$0.68  $0.24   $5.66   $499.32 $13,954
CARRL           55,650  -$0.68  $0.25   $5.61   $499.32 $13,825
Amdocs          51,906  -$0.68  $0.27   $5.69   $499.32 $13,794
#6              55,830  -$0.68  $0.24   $5.63   $499.32 $13,598
#7              60,901  -$0.68  $0.21   $5.43   $499.32 $13,040
#8              48,304  -$0.68  $0.25   $5.83   $499.32 $12,298
#9              56,144  -$0.68  $0.20   $5.32   $499.32 $11,423
#10             90,976  -$0.68  $0.12   $4.84   $499.32 $11,276
#11             62,432  -$0.68  $0.17   $5.13   $499.32 $10,720
#12             65,286  -$0.68  $0.16   $4.53   $224.32 $10,706
#13             64,044  -$0.68  $0.16   $4.99   $499.32 $10,112
#14             76,994  -$0.68  $0.13   $4.91   $499.32 $10,049
#15             54,195  -$0.68  $0.18   $5.29   $499.32 $9,741
#16             79,294  -$0.68  $0.12   $4.47   $249.32 $9,464
#17             51,477  -$0.68  $0.11   $4.00   $111.32 $5,683
#18             30,539  -$0.68  $0.18   $5.34   $499.32 $5,484
#19             50,475  -$0.68  $0.04   $3.44   $99.32  $1,925
#20             42,270  -$0.68  $0.04   $3.64   $99.32  $1,706
#21              1,551  -$0.68  -$0.03  $3.60   $53.32  -$54
===============================================================
*  N is the number of for which the predicted donation amount
   > $0.68
** SUM=sum of (Actual Donation-$0.68) for all records with
   predicted donation > $0.68