Knowledge Discovery Nuggets Index

Users find tangible rewards digging into data mines

Although the term is often confused with OLAP, data mining proves =
its worth in retail and banking

The truth about data mining is as elusive
as the nuggets of information it is designed to find.

Data mining is called the next step beyond online analytical processing
(OLAP) for querying data warehouses. Rather than seek out known
relationships -- such as a list of all catalog customers who recently
moved -- it sifts through data for unknown relationships, such as a
previously unsuspected link between gourmet food purchases and
motorcycle ownership.

'If you say, `How many widgets did we sell in the spring of 1996 in
sales region A vs. sales region B?' that's OLAP,' says Mark Brown,
program manager for data mining at SAS Institute, in Cary, N.C. 'If you
say, `What are the drivers that caused people to buy these widgets from
my catalog?' that's data mining.'

But is data mining still unproven and overhyped? Or is it more
successful than is generally acknowledged and simply kept quiet because
it provides users with a competitive advantage?

Aaron Zornes, executive vice president of the Meta Group, in Burlingame,
Calif., says it's a little of both. The value of data mining is proven,
but it remains difficult to use.

'Data mining is a deep, dark, secret weapon within corporations that is
providing such a competitive advantage they don't want the world to know
what they're doing. But the tools are not easy enough for most
corporations to use,' Zornes says.

MICRO MINES. There is some evidence that Zornes is right. In
most corporations, traditional, server-based data mining remains mainly
in the hands of IS professionals. Client-based data mining that
analyzes a subset of the data-warehouse contents and touts ease of use
is new, and its effectiveness has yet to be determined. Among the major
data-mining players are IBM, Thinking Machines, DataMind, Pilot
Software, Business Objects, SAS Institute, Angoss International,
NeoVista Solutions, Magnify, and Cognos.

Zornes says that high-end, server data-mining software licenses
typically cost $150,000 to $200,000, while desktop or 'micromining'
software licenses typically cost $500 to $50,000.

Although these high fees may put you off, some vendors and users say
data mining offers a clear return on investment and a clear competitive
weapon.

'Data mining is quickly becoming a necessity, and those who do not do it
will soon be left in the dust,' Brown says. 'Data mining is one of the
few software activities with measurable return on investment associated
with it. The banking and catalog industries are making lots of [returns
on investment] today.'

The SAS server-based Enterprise Miner license fee costs $45,000 or
more.

Brant Davison, product manager of business-intelligence software
solutions for IBM's Software Group, in Somers, N.Y., says companies can
most easily use data mining if they've invested in a data warehouse,
although they can get along without one if they're willing to assemble
the data from various database sources. IBM's Intelligent Miner can
deal with data warehouses containing hundreds of gigabytes or terabytes
of information.

IBM entered high-end, server-based data mining a year ago, but its
efforts in low-end desktop data mining -- where it deals with mining PC
spreadsheets containing as much as hundreds of megabytes of data --
remain a research venture. IBM's Intelligent Miner licenses range in
price from $25,000 to $150,000 for systems from Risc/6000 Unix machines
to System/ 390 mainframes.

'I think data mining is taking off, although it's not taking off in all
segments of the marketplace,' Davison says. 'Where a lot of customers
are finding value is in developing models for customer buying behavior.
In particular, we are seeing initial acceptance of data mining in
insurance, finance, retail, and telephone companies. Those industries
have a lot of customers, products, and transactions, and they need a
system to help them understand the value within that information.'

BEHAVIOR. Banks, for example, may use data mining to identify
their most profitable credit-card customers or their highest-risk loan
applicants, Davison says. They also may seek to prevent fraud by using
a data-mining technique called 'deviation detection': Rather than
finding relationships between different groups of data records, it finds
events that are outside the norm that could be a sign of fraudulent
activities.

Brown says companies that use data mining to study their customers
usually are focused on how to retain customers, separate profitable
customers from unprofitable ones, uncover fraud, sell existing customers
new products, and understand why some customers leave.

Davison sees data mining and OLAP working together.

'Using data mining, you may come up with a model to find who are the
most profitable customers. Then you may do more traditional OLAP
analysis of that subset of data to see what the impact would be if you
lost those customers, how it would affect your bottom line,' Davison
says.

But how accurate is data mining? When it comes to analyzing data, it's
pretty accurate, says Alex Moissis, director of product marketing for
Business Objects, which has headquarters in Paris and in San Jose,
Calif.

'But, to the extent to which the tool makes a prediction based on this
data set, you start getting into the issue of accuracy,' Moissis
says.

In other words, there is no guarantee that all gourmet food eaters will
want motorcycles just because the ones in your data set did; 'garbage
in, garbage out' applies to data mining, as well.

Moissis says desktop data mining on a PC strikes at the three major
obstacles to wider adoption of mining technology: the customer
perception that it is too difficult, too expensive, and conceptually
difficult to understand (and therefore difficult for management decision
makers to believe in.)

'The audience for desktop data mining is the mainstream business user,'
Moissis says.

For example, although an automobile manufacturer probably has a large,
server-based data-mining application for corporate use, its individual
auto dealers may need desktop data mining to look at buying patterns for
just their own customers. As a result, desktop data mining will
complement rather than replace server-based data mining and offer far
lower prices to spur adoption.

Business Objects' Business Miner desktop data-mining software,
introduced earlier this year, costs $995 as a stand-alone application on
a Windows-based PC.

REALLY CLEAN DATA. In general, data mining involves refining
data so that it uses the same variables, then searching for patterns in
the data using statistical software models. Users report that preparing
data for mining is frequently 80 percent of the work. Among the
data-mining techniques are 'neural networks' (programs that mimic the
brain's ability to learn from its mistakes), 'time-series analysis'
(year-to-year comparisons), and 'tree-based models' (branching systems
that show relationships in the form of a hierarchy, such as an
organizational chart).

Although banks, financial services, and direct-marketing companies have
been doing something similar to data mining for the past 15 to 20 years,
many have relied on data-service companies to provide them with
predictive statistical models, Brown says. Now new data-mining software
allows those corporations to do the work in-house while coupling
traditional statistical techniques with software-industry technologies, =
such as neural networks and decision trees.

Early users of server-based data mining report that the somewhat
esoteric technology is bringing back some tangible results.

At Pittsburgh's Mellon Bank, IBM's Intelligent Miner is being used with
S/390 and RS/6000 servers and DB2 databases to study as much as 10GB of
information on consumer bank customers, with an eye to retaining the
most profitable ones. Based on historical data, the bank tries to
predict which customers will be profitable in the future. The bank also
uses data mining to project which customers are likely to switch from a
Mellon credit card to another bank's card based on historical patterns
of use.

Data mining showed that the best predictive factors for Mellon
credit-card customer attrition are the frequency of card use and the
types of purchases that are or aren't made, says Pete Johnson, vice
president of Mellon's advanced technology group. The bank tracks broad
categories of credit-card purchases, such as whether they were made at a
grocery store, a department store, or a gas station. Johnson declines
to name the purchase categories that data mining showed were leading
indicators of customer attrition because he considers it valuable
competitive information.

'Data mining helps us play in the national credit-card market more
effectively. That's what all the national banks are trying to do with
their data-mining efforts,' Johnson says. 'Small credit-card companies
that are unable to embrace this technology can no longer survive because
they can't manage their customer bases.'

Although banks such as Mellon have been using statistical models for
many years, 'current data-mining software is more scalable and can
analyze bigger quantities of data. And the engineering of the software
is such that we don't have to write lines and lines of code to do it,'
Johnson says.

ROCKET SCIENCE. But data mining remains complex. Although
Mellon's goal is to have nontechnical business analysts use the IBM
data-mining software, today it is used only by IT people and
sophisticated business analysts.

Fingerhut Companies, a Minneapolis-based direct-mail catalog company
using SAS' server-based Enterprise Miner on IBM mainframes and RS/6000
Unix machines, is sifting through a database of 10 million to 12 million
current customers to find which are most likely to buy products from one
of the company's many catalogs.

Fingerhut, which has 9,500 employees and mails 130 different catalogs
each year, is among the true believers in data mining: All catalog
mailings, credit-granting decisions, and inventory-stocking decisions
are based on it, says Andy Johnson, Fingerhut's senior vice president of
marketing.

Fingerhut wants to find out which customers it could profitably mail
catalogs. It recently used data mining to study past purchases of
customers who had changed residences to see if they had preferences.
Data mining showed those customers were three times more likely to buy
items such as tables, fax machines, phones, and decorative products, but
that they were not more likely to purchase high-end consumer
electronics, Johnson says. Fingerhut used that information to create a
special catalog that it mailed only to those customers who had recently
moved.

Johnson's only caveat: You need good data that has been properly
prepared in order to make money using data mining.

'People who can't see the value in data mining as a concept either don't
have the data or don't have data with integrity. We've spent a lot of
time, money, and energy getting those two things.'

Although Fingerhut has used statistical modeling for about 20 years, new
data-mining software allows the company to look at a broader range of
information and larger databases, says Bill Flach, Fingerhut's director
of marketing analysis and research. For example, before data mining,
Fingerhut's statistical analysis was limited to taking samples of 10
percent to 20 percent of its customers. With data mining, it can
examine 300 specific characteristics of each of the 10 million to 12
million customers in a much more focused way.

Flach believes new data-mining software will be easier to use by people
who are not IS employees or statisticians. But for now, data mining
remains in the hands of about 30 Fingerhut people trained in statistics
plus another 100 IS users.

But even data mining's present accomplishments are impressive, says
Mellon's Johnson. Data-mining results have caused the bank to rethink
its view of transaction data.

'Traditional information-management systems are not designed to collect
transactions as information assets. We didn't know until we did data
mining that transaction detail is very valuable,' Johnson says. 'I like
to describe data mining as the carrot that justifies the expensive stick
of building a data warehouse.'

Steve Alexendar is a free-lance writer based in Edina, Minn.

Copyright © 1997 InfoWorld Publishing Company