Measuring PHI Results: Correlations, Conversions & Causes

Measuring productivity and performance is usually either very simple, or extremely difficult. When workers are already being monitored, evaluated, and paid based on some sort of “piecework” arrangement, their output is already being measured. Translating that into “performance” can be no more than adding a few technical quality and customer satisfaction metrics, to be sure quantity is not being delivered at the expense of quality.

But when there is no system for objectively measuring productivity and performance, the task is likely to be highly complex, expensive, and subject to loads of doubt as to the validity and reliability of metrics. The most common approaches are either self-reported estimates by workers themselves, or subjective appraisals by their supervisors. Both of these are subject to understandable and well-documented biases.

Self-reporting of output and performance are subject to being exaggerated by employees, either through self-delusion or self-protection. Low performers are known to tend toward claiming at least average levels, if not higher. High performers may under-report, for fear they be thought of as “bragging”, or self-congratulatory. When it comes to self-reporting of the effects of health factors on productivity or performance, exaggeration may apply when people feel really bad, or under-stating impairment may be used lest the overall worth of the individual be undervalued.

In one example, where call center agents’ objective productivity was routinely measured, workers affected by health problems overstated their impairment by 2½ times, with actual output only off by 8%, compared to an average self-reported reduction of 20%. [G. Pransky, et al. “Performance Decrements Resulting from Illness in the Workplace” JOEM 47:1 Jan 2005 34-40]

There are at least half a dozen widely accepted and used self-reporting survey options, each of which is likely to vary measurably in results from the other, in addition to varying in difficulty and cost of application. Purveyors of such methods are likely to report high levels of correlation between objective measures and their results, but such a correlation can result from a consistent bias as well as consistent accuracy.

Mathematically, if a self-reported method delivered results that were consistently 2.5 times greater than objective measures, the correlation between the self-reports and reality could be quite high, approaching 1.00, the highest level of correlation available. But that only shows that the survey method varies in a consistent way with actual productivity or performance, not that it comes close to reflecting true values. It would be considered reliable, because of its consistency, but not valid because of equally consistent inaccuracy.

What is needed for validity in such situations is a consistent conversion factor that translates self-reported data into a far more accurate estimate. This is only possible when there is high correlation, showing that workers are consistent in the degree of over- or under-reporting. With such consistency, e.g. if they mainly come very close to overstating their impairment or output by a factor of 2.5x actuality, this factor can be used by convert self-reported data to reality by simply multiplying self-reports by 40%.

Even when productivity/performance is accurately measured, there arises the question of whether changes that are noted therein should be attributed to whatever causes have been initiated to bring them about. While improvements in health have often been found to yield apparent improvements in productivity/performance, there are many other causes that may have contributed, as well.

As is frequently noted in Health as Human Capital (hhcf.blospot.com), a wide range of factors contribute to both normal absences and longer disability as causes of lost productivity and performance. How much workers get paid compared to what they earn when present is one of the biggest causes of differences therein. Improvements in worker commitment to their work or employer, training and development, new technologies and support systems can all cause significant improvements.

Changes in compensation levels and systems can also be powerful causes. When Safelite switched from an hourly pay to a pay-for-performance system, productivity increased by 44% in the very first year the new system was implemented. [E. Lazear “Performance Pay and Productivity” American Economic Review 190:5 Dec 2000 1346-1361]

Increasing employee autonomy and flexibility can have similar impact. Empowering workers to perform their tasks whenever and wherever they pleased enabled Best Buy to improved corporate staff processing of orders by 36% in the first year of the new system, in addition to cutting staff turnover from 16.7% to zero. [M. Conlin “Smashing the Clock” Business Week Dec 11, 2006 (www.businessweek.com)]

If employers choose to offer incentives as part of their PHI initiatives that are partially or wholly based on employee productivity or performance, then the incentives, themselves, may be responsible for improving employee outcomes, perhaps as much as improving their health. The only way to systematically separate the effects of different possible causes is to use control vs. experimental groups, which also reduces the overall impact of the initiative being experimented with. Employers are understandably reluctant to reduce such impact, when they have faith that the initiative will work.

Correlations, conversion, and causes are all important concerns when measuring problems and the effects of solutions in PHI. All can be handled if sufficient care and planning is included in applying ways of improving productivity and performance. Such care is essential to ensure that “results” discovered are real and correctly attributable to the PHI intervention applied.