Research: Not Adding Up: Free Ridership & Spillover Calculations in Energy Efficiency Evaluations

September 24, 2020

In a recent article, “Not adding up: free ridership and spillover calculations in energy efficiency evaluations,” (Energy Efficiency, June 2020), Bloustein School Research Professor Frank Felder and colleagues Zachary Froio (Applied Energy, Red Bank, NJ) and Pranay Kumar (Center for Energy, Economic, and Environmental Policy, Edward J. Bloustein School of Planning and Public Policy), examine a key element of evaluation, measurement, and verification (EM&V) studies for energy efficiency programs involving estimation of net energy savings that account for free ridership, spillover, and induced market effects (FR, SO, and ME, respectively).

The existing literature recognizes these effects to be significant and provides detailed guidelines to estimate them. However, there appears to be a disconnect between these guidelines and field evaluations conducted in practice. Their meta-analysis of 120 studies from 2006 to 2018 indicates that most free ridership and spillover estimates are based on survey results and expressed in percentage terms. The authors note that simply adding these percentages numerically without converting them into a common unit is inaccurate and obscures a program’s true impact.

Additionally, there exists wide variations in nomenclature, classification, and methodologies adopted to estimate these metrics across programs and jurisdictions. A scatterplot analysis of the reviewed EM&V reports indicates that with few exceptions, free ridership and spillover do not necessarily offset each other. A proposed alternative approach is to estimate free ridership and spillover in energy units with costs in dollar terms, e.g., as the difference between a program participant’s total willingness-to-pay and the total financial impact of the program’s existence. They also feel that a consistent, transparent, and reliable evaluation methodology to estimate free ridership and spillover effects across programs and jurisdictions based on randomized or quasi-experimental designs will not only improve accuracy but will also have better comparability for informed policy decisions in the future.

An alternative approach for the analysis of FR and SO is to estimate and report these values in terms of energy units and analyze them in dollar terms to more accurately determine program effectiveness. Based on the dynamic behavior of FR as a function of a program’s presence in a given market, another method would be to estimate free ridership in terms of participant’s total willingness-to-pay for EE measures in the absence of the program. This method may provide a more accurate illustration of a program’s cost-effectiveness in terms of dollars per unit of reduced energy consumption or in terms of dollars per ton of carbon dioxide abated.

As such, FR, SO, and ME should instead be reported in terms of a program’s gross energy savings and associated dollar savings for a more transparent program evaluation design. Additionally, for a meaningful comparison across the measure, program, portfolio, and utility levels, it is recommended that a consistent and reliable methodology be uniformly adopted across all EM&V studies. As a step forward, randomized or quasi-experimental designs can be tried for more accurate impact evaluations and for better comparability of EE programs across jurisdictions.

Recent Posts

Two MCRP Alumni Earn Planning Excellence Awards

The New Jersey Chapter of the American Planning Association (APA New Jersey) has announced the recipients of the 2024 Planning Excellence Awards, and two Bloustein School alumni are recipients. Courtenay Mercer AICP, PP (BA '01, MCRP ' 02) was selected as the Budd...

Election Aftermath: The Path Forward for Democracy

Policy Implications, Representation Milestones, and Strengthening American Democracy in the Aftermath of Election 2024 In a special post-election episode of EJB Talks, Dean ⁠Stuart Shapiro⁠ and ⁠Elizabeth Matto⁠, Director of the ⁠Eagleton Institute of Politics⁠, offer...

Heldrich Report: Examining Student Outmigration in NJ

Where Students Go: Examining Outmigration in New Jersey Building upon previous research on outmigration, a new report measures the gaps in the New Jersey Statewide Data System (NJSDS) that limit the ability to track New Jersey high school graduates who pursue higher...

NJSPL Blog: Who Attends Virtual Schwartz Rounds?

By Irina B. Grafova, Pamela B. de Cordova, Jennifer Polakowski, and Jessica Anderson The importance of well-being programs has never been more critical. Virtual Schwartz Rounds (VSR) provide a platform for connecting with nursing colleagues facing challenging...

Communicating Environmental Issues to the Public

A Conversation with Kati Angarone RU '98, MPAP '04 on Communicating Environmental Issues to the Public In this episode of EJB Talks, host Stuart Shapiro speaks with Kati Angarone RU '98, MPAP '04, currently the Chief Strategy Officer at the New Jersey Department of...

Upcoming Events

Bildner Center: The 25th Annual Rutgers Jewish Film Festival, November 7–21

Regal Cinema Commerce Center 2399 US-1, North Brunswick, United States

The The 25th Annual Rutgers Jewish Film Festival will feature fourteen thought-provoking and entertaining films, dynamic discussions with filmmakers and special guests, and numerous New Jersey premieres on dance, music, […]

Event Series CAREERS

Virtual Career Drop-ins

Virtual

Stop by virtually on Mondays (except for holidays) beginning September 9th through December 16th between 11 am and 1 pm to ask a quick (15 min) career-related question of Bloustein […]