10 ways to assess process performance

In a world full of management jargon about being ‘data driven’ and having ‘accountabilities’ and ‘KPIs’ and ‘dashboards’, it’s easy to think that we know what’s going on with our processes. In practice, however, there are crucial disconnects between our process purpose—what our customers want and what our business needs—and the reports that purport to represent process performance.

These 10 steps provide an overview of what we need to do to build a bridge from process purpose to assessing the adequacy of process performance.

1. Understanding the voice of the customer
Essential question: Who are our customers, and what do they want?

All processes exist to serve a customer, to provide some outputs that are create value for someone. Therefore our first challenge is to understand who are the customers for the process, and what their requirements are. Developing this understanding is known as listening to the ‘Voice of the Customer’.

Customers may be direct beneficiaries of the process, or they may be incidental stakeholders in the process outcomes. Some customers are external to the organisation—either as individuals or organisations—while others are internal, usually in the form of intermediate processes that come between the process-in-focus and the customer. Normally, the direct external customers are the prime focus for the process, but the other groups’ needs must also be considered.

With some customers we may have formal agreements that spell out the contractual requirements. With others, especially in consumer markets, we may need to carry out market research, including interviews, focus groups and surveys, to establish the customers’ requirements and priorities.

The Kano model is a useful way of characterising customer requirements. Kano considers the relationship between how well the process achieves a given customer requirement and how the customer feels about the outcome. There are three types of relationship:

1. ‘Must have’ (expected) characteristics: These are non-negotiable; the process has to deliver them and however well it does so the customer never feels more than ‘so what’. For example, it’s expected that the wheels won’t fall off when you drive your car.
2. ‘Wanted’ (more is better) characteristics: Here there is some sort of correlation between how well the characteristic is delivered and the customer’s degree of enthusiasm. For some customers, the faster a car’s acceleration the better, for other customers the less fuel used, the better. You can segment a market depending on different customer preferences on this dimension.
3. ‘Delighter’ characteristics: Whatever it takes to create customer enthusiasm and loyalty. Innovation can be one way of delivering delighters, so can style and relationships. Marketers love dreaming up delighters; however, customers are sensitive to disconnects between the promise and the delivery.

As a first approximation, accuracy characteristics tend to be ‘must haves’ and timeliness characteristics tend to be ‘wanteds’.

Within the process, the activities that directly contribute to the creation of attributes that the customer requires are called Customer Value Added, or simply Value Added. The focus of Lean improvement efforts is to minimise the time and resources used for non-Value Added activities.

2. Understanding the voice of the business
Essential question: What does the business need?

The customer has certain requirements of the process; the business has certain constraints and objectives as to the resources used to operate the process. Management is the art of balancing these.

Most organisations have a plethora of management reports that account for the efficiency of the business. However, the data in these reports is seldom collected and presented in such a way as to illuminate process performance. Experienced process professionals have learned not to rely on the business’s standard management reports for their data.

The business also has requirements to meet obligations regarding safety, environmental, and legislative requirements, as well as satisfying its internal controls (for example, quality assurance). These requirements are known collectively as the ‘Voice of the Business’. Process activities related to meeting these requirements are known as Business Value Added, and are subject to scrutiny during improvement efforts.

3. Results measures
Essential question: How do we know if we’ve done a good job?

Good process monitoring requires us to track both the attainment of customer requirements—the effectiveness of the process—and the usage of resources to meet those requirements, the efficiency of the process.

This means that we need a suite of measures that cover customer requirements for timeliness and accuracy as well as business requirements for resource utilisation. These measures are known as results measures: we are literally measuring the results of the process.

Customer satisfaction surveys, while important, are too expensive, unreliable, and after-the-event to rely upon. We need to develop proxy measures and periodically calibrate these against the customer experience, bearing in mind that customer expectations change over time.

4. Process measures
Essential question: How do we know if we’re going to do a good job next time?

Measuring the outcomes from the process can tell us whether we’ve done okay. However, the outcomes can’t tell us if there are problems on the horizon, nor can they tell us what to look for when problems do arise.

To be able to do this, we need to measure not just the outcomes from the process, but also the key determinants of successful process outcomes. This means that we need to have a good understanding of the key control factors of the process that we need to manage in order to attain our targets or conversely, what parts of the process we need to measure in order to understand why we are not attaining them.

Different types of process have different key factors. For some, the key factor may be the preventive maintenance and set up of production equipment, for others, it may be the availability of suitably skilled and trained staff, and so on. Using a fishbone—cause and effect—diagram can help surface and prioritise appropriate control factors.

If we have the right set of process measures, then satisfactory outcomes from the process measures should be a sufficient condition to assure successful results.

5. Operational definitions
Essential question: What exactly are we measuring?

Words can be misleading, and measurement is no exception. For example – what do we mean by ‘on time delivery’?

• Where and how does the clock stop and start?
• Whose clock are we using?
• How accurately do we read the clock?
• How do we handle failed deliveries?
• What conditions might exempt a delivery from assessment?
• Where do we store the data?
• How do we aggregate and analyse it?

All of these considerations need to be taken into account. Measurement is itself a process, and as such is subject to variation. The numbers that we see on our dashboards and in our spreadsheets are a compound of the “real thing” that is varying and of the variation in our measurements. How do we know that the variation in the measurement process is not drowning out the variation in the target process?

The challenges are even more acute when we have a subjective assessment process, e.g. of errors in document processing: does the document quality really vary, or is the variation an artefact of inspection inconsistency?

The technical term for describing measurements in terms of the measurement process is an operational definition. Operational definitions are crucial to the integrity of the chain that leads from customer requirements to data analysis.