In a recent article, “Not adding up: free ridership and spillover calculations in energy efficiency evaluations,” (Energy Efficiency, June 2020), Bloustein School Research Professor Frank Felder and colleagues Zachary Froio (Applied Energy, Red Bank, NJ) and Pranay Kumar (Center for Energy, Economic, and Environmental Policy, Edward J. Bloustein School of Planning and Public Policy), examine a key element of evaluation, measurement, and verification (EM&V) studies for energy efficiency programs involving estimation of net energy savings that account for free ridership, spillover, and induced market effects (FR, SO, and ME, respectively).
The existing literature recognizes these effects to be significant and provides detailed guidelines to estimate them. However, there appears to be a disconnect between these guidelines and field evaluations conducted in practice. Their meta-analysis of 120 studies from 2006 to 2018 indicates that most free ridership and spillover estimates are based on survey results and expressed in percentage terms. The authors note that simply adding these percentages numerically without converting them into a common unit is inaccurate and obscures a program’s true impact.
Additionally, there exists wide variations in nomenclature, classification, and methodologies adopted to estimate these metrics across programs and jurisdictions. A scatterplot analysis of the reviewed EM&V reports indicates that with few exceptions, free ridership and spillover do not necessarily offset each other. A proposed alternative approach is to estimate free ridership and spillover in energy units with costs in dollar terms, e.g., as the difference between a program participant’s total willingness-to-pay and the total financial impact of the program’s existence. They also feel that a consistent, transparent, and reliable evaluation methodology to estimate free ridership and spillover effects across programs and jurisdictions based on randomized or quasi-experimental designs will not only improve accuracy but will also have better comparability for informed policy decisions in the future.
An alternative approach for the analysis of FR and SO is to estimate and report these values in terms of energy units and analyze them in dollar terms to more accurately determine program effectiveness. Based on the dynamic behavior of FR as a function of a program’s presence in a given market, another method would be to estimate free ridership in terms of participant’s total willingness-to-pay for EE measures in the absence of the program. This method may provide a more accurate illustration of a program’s cost-effectiveness in terms of dollars per unit of reduced energy consumption or in terms of dollars per ton of carbon dioxide abated.
As such, FR, SO, and ME should instead be reported in terms of a program’s gross energy savings and associated dollar savings for a more transparent program evaluation design. Additionally, for a meaningful comparison across the measure, program, portfolio, and utility levels, it is recommended that a consistent and reliable methodology be uniformly adopted across all EM&V studies. As a step forward, randomized or quasi-experimental designs can be tried for more accurate impact evaluations and for better comparability of EE programs across jurisdictions.