Reliability of metrics influences benchmarking
Facilities managers calculate key performance indicator (KPI) metrics that allow them to track the performance of a facility and assist the C-suite in making budget decisions. Thus, it is essential that these metrics are both valid and reliable.
Metrics are considered valid when they measure what they intend to measure, and reliable when they are calculated the same way each time across every site. Reliable metrics assure facility managers that they can trust the calculations and can compare them across different points in time (in house) and across buildings/organizations (benchmarking).
However, metric reliability can be threatened easily. Consider two commonly used metrics — age of plant (AoP), and the facilities condition index (FCI).
Both metrics are simple ratio calculations: AoP = accumulated depreciation/depreciation expense; and FCI = facility(-ies) maintenance deficiencies/facility(-ies) current replacement value.
However, these two metrics can vary widely depending on the consistency of decisions related to what is included in calculations. For example, FCI reliability may be influenced by decisions about which elements are included in surveys (e.g., leased properties, communications, information technology, elevators and parking structures in calculations), which years are included when forecasting capital maintenance costs, and the use of different estimation tools or sources (e.g., Whitestone Research Corp. and RSMeans).
In addition, calculation decisions related to the size of the organization, such as inclusion or exclusion of in-house or consultant project management and construction team salaries and costs, also influence reliability. Furthermore, AoP reliability may be distorted by the acquisition of facilities (artificially reducing AoP), or variabilities in an organization’s capitalization threshold guidelines (higher thresholds result in increased depreciation expenses and lower AoPs).
These examples illustrate the importance of building an awareness and understanding of how even small inconsistencies in the calculation of KPI metrics (across different buildings, years or organizations) like the AoP and FCI can influence reliability, and underscores the potential need for increased transparency and universal guidelines on KPI calculations.
Lisa Walt, Ph.D., senior analyst, advocacy, American Society for Healthcare Engineering