Introduction
In previous blogs we have discussed the benefits of the
Stonemont KPI and the
Mix Risk analysis tool.
Mix Risk uses the
Stonemont KPI to
provide an evaluation of potential future
performance of a mix design prior to or during production of the mix. However, the Stonemont KPI also can be used
to analyze production results directly and this is done using Product
Evaluation. Remember that the original
mix design meets the specifications and is generally close to the target
values. For the purposes of this
example, we are assuming that the design was put into production without the
benefit of prior information provided by the Mix Risk analysis.
Product Evaluation
 |
| Figure 1. |
Figure 1 shows the results of a Product Evaluation for a
portion of the production run for this mix.
Notice that this evaluation considers properties that are deemed
important for this particular
asphalt superpave product (AC Content, Air Voids,
and VMA). If this were a
concrete
product these properties would differ than those shown. These different properties can easily be
assigned to each product as critical parameters in our software.
One of the most important aspects of the Product Evaluation
tool is that it not only provides you with the statistics necessary to make an
informed decision regarding the performance of your products but it includes a
textural definition of issues that the statistics are revealing. It is clear from this Product Evaluation that
we have issues regarding specifications being inside two standard
deviations. However, the Product
Evaluation will also identify possible trends or changes in mean, distributions that are very off center, large
discrepancies between measured and predicted failures, last values out of
specifications or limits and rounding issues possibly indicative of data
falsification. These textural
definitions help you quickly identify and understand potential issues. This is in contrast to having to manually
scan the data and understand each statistic well enough to identify potential
issues.
 |
| Figure 2. |
Let’s consider air voids (Va@Ndes) in more detail because it
has some interesting characteristics.
The run chart shown in Figure 2 is the same data set of air voids that
was used for the Product Evaluation.
Notice that run chart shows not no obvious trend and no failures
(points out-of-specification) for air voids, which is consistent with the value
of 100% for CTS (Conformance To Specification) shown in the Product
Evaluation. The Product Evaluation shows
a PWS (Percent Within Specification) value of 95.2%. A PWS value above 95% is generally regarded as
sufficiently high enough probability of future conformance. The Ppk value of 0.56 indicates that the
specifications are inside two standard deviations, which is generally a
situation to avoid if trying to meet 95% compliance. The Ppk/Pp value of 0.62 indicates that the
mean of the data is quite off-center relative to the specifications.
These four values are used in the calculation of the
Stonemont KPI, which was 93%. A KPI
value of 93% (below 95%) does provide an indication of future risk of
non-conformance to specifications and could be regarded as an early warning of
potential future problems. Furthermore,
this is an example where simply looking at individual sample results regarding
pass/fail would not have provided a clear understanding of the risk since no
air voids samples failed relative to specifications.
In Figure 3, we have added more samples to the Product
Evaluation and we can see that PWS and KPI have lowered to 93.2% and 90%,
respectively.
 |
| Figure 3. |
By the end of the production run (Figure 4), the air voids
PWS and KPI have dropped to 91.5 and 88, respectively. The run chart of air voids (Figure 5) shows
that we had 5 failures following our initial Product Evaluation. This example
demonstrates that although we didn’t have any previous air void failures that
the risk or probability of non-conformance identified by the Product Evaluation
was real and should have been addressed.
The Product Evaluation results also shows that other critical
parameters, including the ½” sieve, are sources of potential risk. Recall that the Mix Risk analysis we
performed identified an increased risk of producing non-conforming material due
to the ½” sieve.
 |
| Figure 4. |
 |
| Figure 5. |
Early Warning
System
A very powerful feature of
Stonemont Software is the ability
to have Product Evaluations run automatically (Enterprise/Hosted Editions) so
that they can serve as an “early warning system.” This can be done either on a product basis
using the Auto Analysis capabilities or on a plant basis using Auto
Evaluation. They can be run daily,
weekly, or monthly (or hourly in Version 7) and automatically emailed to the
appropriate personnel.
The Auto Evaluation can take advantage of user-defined
triggers as to whether or not a potential issue exists. For example, a trigger of 95% can be placed
on KPI, meaning that if any of the parameters being included in the analysis
for that product are below 95% then the product will be included in the auto
analysis report. In our example using
air voids, the KPI fell below 95% on June 8th so we could have been
alerted to the potential issue prior to any actual failures. Although the auto evaluation can be run for
every product at every plant it can be setup so that only those products that
indicate a potential issue will be reported.
This reduces the amount of data that a quality manager must filter through
to identify those products that require their attention.
Summary
Hopefully we have shown the importance of using both Mix
Risk and Product Evaluations that incorporate the Stonemont KPI . Mix Risk will help identify potential issues
in the mix design stage prior to them becoming problems during production. Product Evaluations will monitor the product
performance during production hopefully prior to them becoming customer issues.
James Beal
Adrian Field
