Beyond “Does It Work?”: Laura Peck on Policy, Evidence, and Impact
EJB Talks returns for Season 14 with Dean Stuart Shapiro speaking with Laura Peck, one of our newest Public Policy Associate Professors and a Principal Faculty Fellow with the Heldrich Center for Workforce Development. Laura discusses her journey from undergraduate activism to policy analysis, a PhD, tenure at Arizona State, and more than a decade at Abt Associates. She shares how that work shaped her focus on careful, policy-relevant research, specifically taking about her evaluation of the federal Health Profession Opportunity Grant (HPOG) program. Laura emphasizes that the real question of her research isn’t just whether programs work, but really how and for whom they work. With her return to academia at Bloustein, she is looking forward to getting back into the classroom to help students understand how programs run in the real world and whether they’re actually helping the people they’re meant to serve. Her goal is to give them thetools to figure out what’s working, what needs fixing, and how to make smart decisions about public programs in government and nonprofit spaces.
Transcript
Stuart Shapiro
Welcome to EJB Talks. I’m Stuart Shapiro, the Dean of the Bloustein School. And the purpose of this podcast is to highlight the work my colleagues and our alumni in the fields of policy, planning, and health are doing to make the world a better place.
Today, unbelievably, we are beginning our 14th season of this podcast with a discussion with our new professor in the Public Policy Program, Professor Laura Peck. Welcome to the podcast, Laura!
Laura Peck
Hi, thank you so much for having me!
Stuart Shapiro
It’s great to have you here. And when guests are here for the first time, we almost always start with an origin question. How’d you get into public policy?
Laura Peck
How did I get into public policy? Well, as an undergrad, I was a bit of an activist. Maybe not surprisingly, volunteering for PETA, protesting the Clarence Thomas nomination with fellow feminists, that kind of thing. And I decided that grad school in public administration and public policy would be the right way to put those activist tendencies to productive and practical use.
Stuart Shapiro
Excellent, excellent. I remember the Thomas hearings.
Laura Peck
Now people can figure out how old I am.
Stuart Shapiro
((laughing)) Yeah, same! And 36 years later, he’s still on the court! So… 33 years later, actually.
So, you’ve had a very interesting career path. You were an academic at Arizona State. You went to consulting for a while, and now you’ve come back to us in the university. Can you talk about the time you spent in consulting, your experience, and your decision to come “back to school,” if you will?
Laura Peck
Yeah, of course. So, after my master’s degree, I worked for a while as a policy analyst. And it was that work that really compelled me to go back to school for a PhD, during which time I followed what is probably the more regular track to enter an academic position. And the in fast-forward several years that I had earned tenure, and I served as our associate dean for honors at ASU. And I really enjoyed that work. But also had an opportunity to kind of, reinvigorate my research agenda by working for a company called Abt Associates that does contract research for predominantly the federal government.
And in the process of doing that, I kind of got hooked. I thought I might be in that position for just a couple of years, and it ended up being a dozen. And what really hooked me on working in that sector was the generous funding. The policy-relevant questions that I had an opportunity to weigh in on. And also, really, the opportunity to be methodologically creative. And I did this in partnership with my project officers in the federal government. And when people think about kind of government employees, sometimes I feel like they might use the word “bureaucrat” as a bad word almost. But I give it a capital “B” and an exclamation point and smiley face, because I feel like the people that I got to build relationships with in the process of leading research for government, were really well-meaning and smart. And really great partners in doing interesting and important research.
So for me, it was those connections and the influence that I felt like I had through them that kept me there as long as I was. And then to the other part of your question, how did I make this decision then, to transition back into a university setting? Well, I also have really valued the opportunity to teach and kind of proliferate the field, if you will, through teaching. And so, I am now really excited to be able to continue doing that important and policy-relevant research. But also in combination with training and nurturing future policy analysts and public servants.
So for me, I think that Bloustein is a really great home because all of our faculty aim to do this good, kind of, applied research that will matter to planning and policy decisions. But then we also have really motivated and energetic students that I am excited to help jettison into their public service careers.
Stuart Shapiro
Great. Well, you warmed your way into my heart there by speaking kindly of bureaucrats. Much of my research is on the federal civil service and regulators and regulations. And yes, I find it hard to control myself as I see what’s going on to many of those people right now, in the civil service in particular.
Also, I reacted to what you were saying in thinking that, it used to be common to move back and forth in and out of academia. And it’s become much less so. So, it really is sort of an interesting path that you’re taking there, and one that, I think, that used to be more common than it is now.
Laura Peck
And I do hope that it brings something of value to our students and to my colleagues too. It’s a different set of experiences and perspectives that I think are relevant to policy studies.
Stuart Shapiro
Well, and I would expect some of our students make the decision you made after your master’s degree and go into, whether it’s consulting, or whether it’s public service, or whatever. And I think your experience will really help them, if the ones that decide to make that step.
So, a lot of your work has been on program evaluation, particularly in workforce or labor policy. Laura’s affiliated with our Heldrich Center for Workforce Development, I should note. And one quote in one of your articles really grabbed me, which was, do social programs work? And a lot of your work is about just that question. And so that’s an important question, right? I’ve referred to the attacks on civil service and on government. A lot of it is because people, often mistakenly, think government programs don’t work. So how do we answer the question, do they work or not? Because the answer’s not always yes, either.
Laura Peck
Right. It’s very rarely an easy yes or no.
Stuart Shapiro
Right.
Laura Peck
There’s usually a lot more that we can learn from the kind of research that we do. And I think that question, I feel like, I think I posed that in a 2003 American Journal of Evaluation article where I was sort of, positioning the common question of do social programs work? Up against a kind of dig deeper set of questions, which is, how do they work and for whom? And so in general, I have tended to go about answering the do they work question by also asking how and for whom.
Most of my research has really been about impact evaluations of U.S. social welfare programs. And I have led a whole slew of large-scale evaluations of U.S. social welfare programs and kind of, labor and workforce development, public assistance, and housing. And in those studies, we have used an experimental evaluation design that involves randomizing people into a control group that represents the status quo. That is whatever we’re doing anyway, or into a treatment group of individuals who gain access to this new program. And so, by looking at the differences in the outcomes between those experiencing the status quo and those experiencing the new version of the intervention, we can get a really good and causal estimate of the program impact. And this is the difference that the program would make in the world above and beyond what else is happening?
Stuart Shapiro
Great. Are you often finding yourself measuring a program’s success against its own goals or against other criteria that you feel you need to apply to it?
Laura Peck
Well, establishing what the program’s goals are is a really important way of judging its success, right? You don’t want to assess a program’s success by some metrics that don’t make sense for them. But there are also, some of the programs that I’ve involved have, as I mentioned, have been federally funded. And the federal government may have a different goal or target for a program that it’s funding than the on-the-ground around program administrators. It’s not usually different, I should actually correct myself. They won’t necessarily have a different goal, but they might have a longer term goal. Whereas the program administrators themselves will have more proximal goals, like first we need to do this, that, and the other to be sure that our program is operating in the best way possible. It’s the strongest version of itself so that we can then subject it to this outside evaluation to assess whether it’s meeting, like, the big social goals.
Stuart Shapiro
Great. So, tell us about your favorite program evaluation or any one you want to talk about.
Laura Peck
Again, I can’t pick a favorite. I love all of my children ((laughing)).
Stuart Shapiro
((laughing))
Laura Peck
But one that I would love to highlight, because I feel like it did so very much, it gave us so much to think about, both substantively and methodologically, is this evaluation of the Health Profession Opportunity Grant Program that the U.S. Department of Health and Human Services funded starting in 2010, I think, is when the first round of grants for these programs went out. And, it’s a program that funded career pathways-style training within the healthcare sector. And this is a sector where there are known to be worker shortages. That is, there’s a high demand for labor. And also plenty of credential-related opportunities that give people an opportunity to advance. And so, they wanted to know whether this new program, called HPOG, was any better than the standard kind of job training that was available out there.
The evaluation itself randomized almost 14,000 people to gain access to this new version of the program or experience the “business as usual” kind of training. And to me, one of the most interesting things about this evaluation was that it took place in 42 programs across 23 grantees in 19 states. And that implies a lot of variation in the program’s design and operations. And so, in the study team, we leveraged that variation and also supplemented it with some additional planned variation to focus on the program components of what was otherwise a multifaceted program to try and identify if there are particular kind of key ingredients to the program’s success.
So we learned that, at least for HPOG, there isn’t really one magic bullet element to this kind of intervention, but instead the holistic nature of the program is what led people to increase their education, earn credentials, and become employed at a greater rate than they otherwise would have in the healthcare sector. It also turns out, I’ll get to the kind of where this leads for the future, because it turns out that even though there was this really important shift toward employment in the healthcare sector, it turns out that for people who come to this kind of program who are unemployed or underemployed with low skills, it’s going to take more than an entry-level credential to help them move up meaningfully in the labor market. And so, I think that that’s kind of where I’m hoping to go in my next phase of research, just to think about the next phase of this kind of intervention and how it can kind of fulfill the promise of the career pathways framework. Which is about successive steps of training, education, and advancement. We got to observe through this evaluation the first step, and I think there’s more to do.
Stuart Shapiro
And do you find that the people running these programs are receptive to the evaluations? And you can use this one as a particular evaluation. Do they learn from them?
Laura Peck
Yeah, so what was particularly interesting about this, I mentioned there were many sites and the… even though it was written into the grant agreement that the federal government had with the grantees operating these programs, that they cooperate with the evaluation. We also know that they, the program administrators don’t tend to love the idea of random assignment. That is randomized people not to get access to their program because they know in their heart of hearts what they’re doing is wonderful and they want to give it access to everyone. But the… It turns out that is an open question. And in order to sort of, endear ourselves, to the program staff, rather than randomizing in a one-to-one ratio, we randomized two people into treatment for every one that went into control. And that we also gave them, each site, a few wild cards. Where if there was someone they really, really wanted to get into the treatment group, they could use a wild card to circumvent that process. And I have to say that in the end, programs didn’t really use very many wild cards. ((laughing))
But it’s interesting when you think about what it actually takes to put one of those kinds of evaluations together in the field across that many locations. There’s a lot of work that our team did with the administrators and program operators on the ground to help them integrate the kind of evaluation into their systems. They had access to kind of a data management system that allowed them to collect data for their own monitoring. And so that’s certainly a benefit to them as well, that they gained access to the data for their own site. But then the kind of aggregated data across all of the sites that really represents the impact of this funding stream of which they were a part.
Stuart Shapiro
Great. All right, so I want to bring this to the teaching a little bit. You’re teaching for the first time this semester with us. When you teach program evaluation or the sort of courses that make the necessary prerequisites to that, what do you want students to get out of it? What do you want them to be thinking as they come out of your classes?
Laura Peck
Well in evaluation in particular, I’ll just focus on that class, which I’m really excited to be teaching for the first time in fall of 2026. So, plug to those listening who want to sign up for that! It’s my hope that class is going to include the “soup to nuts” of evaluation, including both qualitative and quantitative approaches. Because even though I have tended to oversee these impact studies, all of these studies also include implementation research. And that’s a really important part of evaluation, is understanding both the implementation and the impact of the policies and programs that we’re studying.
So it’s my hope that students will come out of this class ready to be top-notch evaluation researchers whether for government or nonprofit programs. And that they can then, from that training, help figure out how we can ensure that our investments in these programs are well spent. That they can help figure out if programs are successful, how to scale them, or if they’re not yet successful, how to retool them so that they can become so.
Stuart Shapiro
Great. And let’s wrap up back to your research. Where do you see it going? What are your plans, short, medium term?
Laura Peck
Well, I am very much looking forward to continuing my research focus on the effectiveness of U.S. social welfare programs. And the kind of, methodological approaches that we take to doing that work. I’m also kind of a bit of a subgroup junkie. ((laughing)) And this really stems from my observation that the average impact of any given program is the combination of a whole slew of varied impacts that accrue to various kinds of subpopulations. And so, I think it’s really important that we pay attention to the subgroup and be careful in how we define who they are and use smart and creative approaches to ascertain how this variation in impacts operates across diverse groups.
And so, I think that there’s really a lot of work still to be done in that field to help us figure out across kind of, heterogeneous populations, how can programs be more effective for the diversity of people that they serve. And so, that’s kind of overall impact, subgroup impact, kind of in the workforce development and kind of public assistance arenas. That’s the kind of work that I am doing and look forward to doing more of.
Stuart Shapiro
Fantastic. Laura, thank you so much for coming on today!
Laura Peck
Well, thank you for having me. And more importantly than the podcast, thank you for hiring me, because I’m really happy to be part of the Bloustein community!
Stuart Shapiro
And we are very lucky to have you! A big thank you as well to Tamara Swedberg and Karyn Olsen who help get this podcast up on the air. We will be back in a week or two with another episode with experts from the Bloustein School. Until then, stay safe!
