SQL/MX Data Mining Guide

Mining the Data
HP NonStop SQL/MX Data Mining Guide—523737-001
4-11
Monitoring Model Performance
Monitoring Model Performance
When measuring a model, consider these questions:
•
How accurate is the model?
The accuracy of the model can be measured as a whole. For example, you can
determine the percentage of records that are classified correctly. The accuracy of
the parts of a model can also be measured. For example, in a decision tree, each
branch of the tree has an associated error rate.
•
To what degree does the model describe the observed data?
The model should be sufficiently descriptive with respect to the observed data to
make clear why a particular prediction was made.
•
What is the level of confidence in the model’s predictions?
Confidence is a measure of how often the model predicts the goal in the training
data set.
•
Is the model easily understood?
A predictive model that consists of a few simple rules is preferable to a model that
contains many rules, even if the latter is more accurate.
However, in the end, the only true measure of a business model is its return on
investment. In a marketing application, measuring a model requires setting aside
control groups and carefully tracking customer responses to various models.