How ACE and ASAP Programs Boost Completion Rates

[music]

0:00:11.5 Speaker 1: Hello and welcome to Office Hours with EAB. Our guest today is Donna Linderman, who is currently the Senior Vice Chancellor for Student Success at SUNY, the system of public colleges and universities in the state of New York. Before joining SUNY, Donna was one of the driving forces behind the development of the popular ASAP and ACE programs at the City University of New York system. The ASAP and ACE programs have been so successful at New York colleges and universities, that they have emerged as perhaps the most intensely studied and most promising models being replicated across higher ed today to improve college completion rates. Donna offers a fascinating glimpse into how the programs work and how they’ve been scaled and adapted to meet the needs of both two-year and four-year institutions across seven additional states so far. So give these folks a listen and enjoy.

[music]

0:01:11.5 Heather Darrow: Hello and welcome to Office Hours with EAB. My name’s Heather Darrow and I’m a Senior Strategic Leader at EAB focused on helping colleges retain students, keep them on path and get them across the finish line with a degree in hand as efficiently as possible. One of the perks of my job is that I get to meet and learn from people who are among the best in the business at managing and improving student success. And luckily for all of us, one of those people is joining me today on the podcast. Our guest today may not be a household name yet, but her work has been studied and imitated widely with her enthusiastic consent. We’re delighted to welcome Donna Linderman, Senior Vice Chancellor for Student Success at the State University of New York, otherwise known as SUNY. Donna, welcome to the podcast.

0:01:58.5 Donna Linderman: Thank you. Delighted to be here.

0:02:01.5 HD: Thanks. For those of you who aren’t familiar with Donna, she’s probably best known for her work in establishing and leading a couple of groundbreaking projects at the City University of New York, known as CUNY. Before she was invited to lead a similar effort at SUNY. Now we’re gonna throw a couple more acronyms at you today, so if you can lock those into your mind early, it’ll save us a lot of time. Donna was instrumental in creating the Accelerated Study and Associate Programs, known hereafter as ASAP. And its related baccalaureate program Accelerate, Complete, and Engage, more commonly referred to as ACE. In a nutshell, both programs target Pell eligible students and provide supplementary financial assistance, as well as comprehensive personalized advising, academic support, and career development assistance. These programs have dramatically increased student completion rates, narrowed opportunity gaps, and helped these institutions gain significant cost efficiencies and benefits. Since their inception in 2007, ASAP and ACE have been studied and evaluated by independent education economists and researchers, and the programs have been replicated so far in seven states. Donna, can you tell us in layman’s terms, how the programs work and maybe share a bit of backstory in terms of what went into getting the programs up and running at CUNY?

0:03:26.5 DL: Sure. So, ASAP and ACE are comprehensive programs that are aimed at removing the barriers to students enrolling in their degree programs full-time, gaining and maintaining academic momentum, feeling fully supported, and graduating on time. So it’s really about helping the students get on track and to move towards the reason that they entered college, which is to earn a degree. So that is easier said than done. And many students that enter college, particularly at public institutions, struggle in one way or another with earning their degrees in a timely manner. So when ASAP began back in 2007, and ASAP was implemented before ACE a number of years later, at CUNY we had very high enrollment at our community colleges, booming enrollment. But every year, large numbers of students that entered our community college were not retained and did not complete their degrees, despite the fact that everyone that entered the door planned to get a degree, and in many cases, they planned to transfer and eventually pursue a bachelor’s degree. And at CUNY at that time, 2007, there was a lot of great work going on across the university and nationally at helping students feel more engaged, providing them with a support here, a support there, organizing their first-year experience.

0:05:00.5 DL: But in isolation, none of these various initiatives and were collectively or individually helping the students graduate in a timely manner. So it was a unique moment in time when our then chancellor, Matthew Goldstein and our then mayor of New York City, Michael Bloomberg, really had this moment of synergy. The chancellor really wanted to dramatically improve our community college completion rates, which troubled him greatly. Their low numbers were consistent with national figures, hovering at about 15 or 16%. And the mayor who really wanted to invest in a few initiatives that were comprehensive, that would help more individuals move into jobs that were family sustaining and helped to move them up and out of poverty. So it was kind of this marriage of the mayor’s economic development goals for the city of New York. Coupled with our chancellor’s desire to really make a difference in community college completion rates. Mayor was prepared to make some big investments through what was then called the Poverty Commission, which became his Center for Economic Opportunity, where they were prepared to make big investments in initiatives, not just at CUNY, but across city agencies that would hopefully move New Yorkers up and out of poverty.

0:06:18.5 DL: So in the case of CUNY, large numbers of students that enrolled in our community colleges were very low income. Many of them were the working poor. They had jobs, but weren’t making family sustaining wages. So ASAP was really an attempt to pull together a comprehensive set of initiatives that removed the financial barriers to students studying full-time by creating comprehensive wraparound support programs and organizing their course-taking experience. Removing some of the financial barriers such as transportation, textbooks, any gap needs that might exist between their financial aid and their tuition and fees, and really structuring their experience in a way that helped them feel engaged and supported through one full-time advisor who would work with them the entire time they were in the program. So it was really attempt to pull together a comprehensive set of experiences and resources that would help the students stay enrolled full-time and graduate in a timely manner. And it was all tied together with this idea of using data in a very integrated manner for both program management and evaluation purposes. So we would really know what we were doing on a moment to moment basis. So it was a unique moment in time that allowed us to create this very comprehensive program. And one of the other things that was very special about the launch of ASAP is that all of our then six community colleges were involved in the project.

0:07:45.5 DL: That there wasn’t something that we piloted just at one or two colleges. All of the CUNY community colleges were involved in it. So we could really see if this very structured model would work for students across our campuses. So it was a very special moment. In a nutshell, what the ASAP program is, it’s a set of comprehensive wraparound support services that help students gain and maintain academic momentum, enhance their sense of integration and sense of belonging at their campuses, and provides them with a set of timely and relevant supports to help them move towards timely graduation.

0:08:23.5 HD: Wow, thank you so much. And that definitely covers sort of the expectations for the program at the outset. Did you get the feeling that you were embarking on something that could be truly transformative for the CUNY system? And what were the biggest concerns that you had about your team’s ability to implement the vision?

0:08:41.5 DL: So that is an excellent question. So if you look at ASAP, just on a piece of paper, if you look at and say, oh, providing Metro cards, paying for textbooks, creating a cohort experience, assigning one advisor with a reduced caseload, integrating academic support services and career development services, early engagement… None of these things on the surface look especially novel and revolutionary. The real magic comes when you marry them together into one comprehensive and very, very carefully managed and implemented program. So at the beginning, I don’t think when you, again, when you looked at the individual elements, many people would say, well, what’s so special about this? This is real common-sense stuff. It was really about the way we managed the program and accepted kind of the challenge, if you will, to really use data on a day-to-day basis to see, are we meeting with students twice a month? For example, with advisement, who’s coming and who’s not coming and why? How can we tweak our blocked courses in the first year where students take at least a couple of courses together in a way that it’s the most meaningful experience. How can we promote students taking winter and summer courses to encourage and help them gain and maintain academic momentum? It was more this kind of the connective tissue, if you will, that is really what made it special.

0:10:10.5 DL: So I would say probably about two years in when we started looking at the data on retention, and academic momentum, we realized we were onto something special. At the time ASAP began, CUNY, and this is the case of community colleges across the country, we didn’t look very carefully, for example, at our two-year completion rates. We were focused on helping students get out in three years. We saw at the two-year mark that about 30% of our students were graduating, which is remarkable because we were not recruiting students that were different from students who would normally enroll at the community colleges. They were very representative, if you kind of looked at a profile of who the first and second cohorts of ASAP were, they were very similar to the students that were enrolling at these colleges more generally. So that’s when I think we really started to realize we were onto something, was when we saw those early indicators of very strong retention and academic momentum.

0:11:06.5 HD: Wow, and perfect segue because my next question focuses on how did you measure success? And if you could talk a little bit about some of the findings that came out of the studies and evaluations that were conducted by those independent researchers.

0:11:19.5 DL: So right out of the gate, we established some pretty clear and stringent program benchmarks, whether it was about program participation or the expectations that we were establishing for retention, credits attempted and earned, and movement towards graduation. We actually created a marker to measure how students were doing in terms of moving towards timely completion. So first and foremost, we created kind of a standardized data management system, where all of the college programs that were participating in the program knew, hey, this is the expectation, all students need to be seen. At the beginning, it was students had to be seen twice a month, regardless of kind of how they were doing in the program. There was the expectation that all students were enrolled in a minimum of 12 credits a semester that they were earning, certain number of credits to move them towards three-year completion. We established, we kind of backed into retention rates where we said, hey, if we wanna graduate at least 50% of our students within a three-year timetable, this should be the retention that we’re working towards on a semester-by-semester basis.

0:12:23.5 DL: If we wanna see the students being successful in their courses, if a student has any remedial need, they need to take those classes immediately and continuously. So we were very regularly kind of looking at where do we wanna be, and then what are the markers that we need to be measuring and on what timetable, and then created those as program benchmarks, that were very clearly communicated to all of our campus partners. So we had a schedule of when we collected data. We had a schedule of when we produced reports that we would look at together. So everyone saw each other’s data. We would create kind of a standard dashboard or table, if you will, where everyone would know, okay, this is the month where we look at the number of students who still have remedial need and who is and is not enrolled in a class and why, and actually discuss that. Let’s look at whether or not students are coming in and seeing their advisors. So this is the type of thing that we would really dig into as a group. The program was structured as a consortium model between the CUNY Office of Academic Affairs and the team that reported up to me and our campus program partners. So it was very transparent. There was not an us and them kind of mentality. We kind of were all in on it together.

0:13:50.5 DL: That doesn’t mean it was always perfect. Obviously, campuses sometimes would push back about this or that, we would push on this, but we got to the place where it was really something that was seen, as something that belonged to everyone. It was everyone’s responsibility that we were moving towards the benchmarks that would help us realize this very, very ambitious three-year completion rate. Again, at the time CUNY, ASAP began in 2007, CUNY’s overall three-year completion rate at our community colleges was only about 16%. And even for students who entered fully proficient with no immediate needs whatsoever, it was only 24%. So setting a target to hit a 50% graduation rate seemed a little bit crazy at the time, but our chancellor believed in it. Those of us that were implementing the program believed in it and we created the structures to make that possible. And regarding your second question, which I’m very proud to talk about. So I alluded a few minutes ago to the fact that about two years into the program, ASAP was originally funded as kind of a three-year pilot program by the mayor’s office.

0:15:00.5 DL: We were given funds to support one cohort of students for up to three years. So we realized two years in that we were doing remarkably well with about 30% of students graduating at the two-year mark, very, very high one and two semester retention rates and strong academic momentum. So right out of the gate, we were putting those data in front of the city. And they very quickly decided, okay, we’re going to baseline your funds, which that’s New York City code for. This is now a permanent allocation to CUNY to continue to run this program. We brought a second cohort of students in. They did equally well, at the three-year mark, we not only met, but we exceeded our three-year graduation timetable. About 54% of students graduating at the three-year mark. 80% of students retained after one year. So it was really quite remarkable. And when we disaggregated the data and looked at it across campuses, across student groups, all student groups were making gains. So it wasn’t just that one or two groups were being successful and others were not. We still saw some opportunity gaps, but all student groups were at or above where we wanted to be. So because of those early outcomes generosity of the city of New York that baselined our funding, CUNY entered into a first phase expansion of the program where we went out and built on the city’s investment and raised some additional private dollars.

0:16:37.5 DL: We also decided to build our evidence base by entering into a random assignment study with MDRC of our work at CUNY. And we did a series of cost studies with the Center for Benefit Cost Studies and Education at Teachers College to look at both the cost effectiveness and the cost benefits of the program. So the early findings from all of that additional research demonstrated equally important and kind of validated what we were seeing within our own analysis of the program, which is pretty rigorous on its own. But MDRC study really solidified that this program really was demonstrating remarkable outcomes of helping students who would not otherwise be successful, graduate at a pretty much double the rate of their counterparts.

0:17:28.5 DL: So we continue doing all of our internal quasi-experimental much comparison group analysis of ASAP as we grew the program. We entered into a random assignment study with MDRC on which looked at a treatment and control group. The outcomes from the teacher’s college study found that the program was very cost effective when you looked at the cost per graduate. ASAP was significantly less expensive because so many more students graduated. And then when you looked at the cost benefits of the program, the additional investment, Hank Levin and his team found that for every dollar invested by the taxpayer, $3.50 was realized in terms of increased tax revenue and savings to the public sector in terms of public assistance, criminal justice, and health expenses.

0:18:17.5 DL: So it was very, very rigorously evaluated and going another step further, the replication demonstration of ASAP in the state of Ohio also led by MDRC in terms of an additional RCT, very much mirrored the outcomes found at CUNY in terms of the program essentially doubling the graduation rate of students. So very rigorous evaluation on all levels. I’ve neglected my dear ACE program are the baccalaureate model in 2015 because of the remarkable outcomes of ASAP. A couple of our very visionary four-year college leaders specifically at John Jay College, Jeremy Travis and Jane Bowers, the then president and provost said we would like to try to replicate this ASAP model within a baccalaureate context.

0:19:08.9 DL: John Jay is one of CUNY’s wonderful four-year colleges that serves very diverse and very low-income population of students. We wanted to see if the model could work in a baccalaureate context. It’s the exact same program. Students have four years to complete their bachelor’s degree with these additional wraparound supports. The model worked equally well in that context, both in terms of our quasi-experimental analysis and in a recently completed randomized control trial of ACE, doubled the completion rates of students who entered the program compared to control group counterparts. So really rigorous analysis demonstrates this model does work. It’s not just me saying so. We have the data to back that up.

0:19:52.5 HD: Yeah. Well, you’ve definitely touched on the expansion of the original projects. Can you talk a little bit about what’s happening with the SUNY system, the expansion of those original projects to SUNY and beyond, and maybe start by telling us what you brought from CUNY to SUNY and talk about where you are in the process at SUNY today?

0:20:12.5 DL: Absolutely. So exactly a year ago, Chancellor King asked me to join his leadership team to create a new role, senior vice chancellor for student success, and to create a unit that really looked holistically at the entire student experience from recruitment into and through our various degree programs here at SUNY. And one of the things that we were very excited about was the possibility of replicating ASAP and ACE and this proven evidence-based model within the SUNY context. We were incredibly fortunate that in last year’s budget, Governor Hochul and the legislator made a major investment of $75 million in a SUNY Transformation Fund, which included $60 million that allowed for us to invest in evidence-based transformational practices across our 64 colleges. And all of the colleges had the opportunity to look at a menu of allowable uses. Whether they wanted to zero in on retention and completion through replicating ACE or ASAP or workforce development programs. And we’re very pleased that 25 colleges stepped forward and said, we would like to replicate ASAP and ACE. So through a structured planning process, these all colleges put a plan forward, but the 25 colleges that opted into replicating ASAP or ACE, 13 of our community colleges and 12 of our baccalaureate granting colleges.

0:21:41.5 DL: Starting this past fall, entering into a process of going through technical assistance, very generously provided by my former team at CUNY, the CUNY ASAP and ACE National Replication Collaborative support systems and districts across the country who wish to replicate ASAP and ACE with fidelity. We’re incredibly grateful to them for working with our 25 colleges to help move them through a process of fully understanding the model, both from its individual elements to how they’re stitched together. My office has been supporting all the colleges in terms of helping them have standardized marketing and messaging to understand a couple of things that look a little different within SUNY. So for example, there is no subway in many places outside of New York City. So the MetroCard was translated into a $50 monthly incentive that could be used for gas or groceries, for example, something that we learned from our Ohio friends. So my office has been helping, as the colleges are learning about the model, begin to create the SUNY version of this. So all of the colleges, I’m very happy to report through their technical assistance, they all launched pilot programs this spring, just started a couple of weeks ago.

0:23:00.5 DL: So we have pilot programs at 25 colleges of anywhere from 30 to 80 students. And the colleges are working towards hitting 150 student total enrollment by this fall. So very grateful that the transformational fund from the state is allowing us to launch these programs. So SUNY now represents, in terms of the number of colleges, the largest ASAP or ACE replication site in the country. Across all of our colleges, just under 4,000 students will be participating. And we will be as rigorously evaluating the program as CUNY did. I actually, just an hour or so ago, came from a meeting where we’re talking about the evaluation of the program. So we’re creating a similar standardized data structure, program benchmarks, and support model to help all of our campuses implement their programs successfully.

0:23:55.5 HD: That sounds like you’re off to a great start. Very exciting for SUNY. They say that invitation is the sincerest form of flattery and I know that exporting the ASAP and ACE programs beyond the state of New York has been an important focus of your work. You’ve already sort of alluded to what CUNY is doing. Can you talk a little bit more about the growth of these programs outside of New York State and the ways that CUNY is supporting those efforts?

0:24:20.5 DL: Sure. So just to be clear, while I no longer work for CUNY, I’m happy to just point to a few amazing things that the former team I was part of is doing. And if I played even a tiny role in any of them, I’m very proud and grateful for that. So the earliest replication efforts of ASAP were in the state of, first the work in the state of Ohio that demonstrated this model could work outside of New York City. So based on those early findings, the program then expanded out to additional states. So there’s been replication in the state of California. In New York state, actually one of our SUNY colleges dipped its toe into the ASAP waters back in 2018 through this early replication work that CUNY was leading. Westchester Community College replicated ASAP in their Viking Roads program, and they’re just completing their RCT through MDRC and the early outcomes are spectacular and extremely impressive. So if anyone wants to see just a little tidbit of what that looks like in a more recent study, I encourage folks to look at MDRC study of the Viking Roads Program at Westchester Community College, the state of Tennessee in Nashville, Pennsylvania through work in Philadelphia, West Virginia.

0:25:45.5 DL: And most recently, North Carolina is the first state to be replicating the ACE model, looking specifically at the community college to senior college transfer. And then again, I claim truly no credit for this, but I’m incredibly proud of the CUNY team. CUNY and SHEO have recently joined forces and they’ve created a kind of building on the early work that CUNY was doing, a national ASAP and ACE national replication collaborative where additional states are entering in to learn more about how they can replicate ASAP and ACE within their jurisdictions and to make it their own. So I would encourage folks to go to the CUNY ASAP and ACE replication website to learn more about that new work which just recently launched. And then of course the tremendous support they’re providing to us here at SUNY. We’re so very grateful to them.

0:26:41.5 HD: I was just about to ask you if you’ve had any advice for other higher ed leaders who were interested in standing up an ASAP or ACE program at their institution or university system.

0:26:51.5 DL: So I would definitely, the first stop would be CUNY ASAP National Replication Collaborative website. They have a wealth of resources there about what their technical assistance looks like and I can’t say enough good things about them. I would encourage anyone that’s interested in ASAP or ACE to take some time just to read some of the studies, just to understand the comprehensive nature of the model. This doesn’t happen so much now, but when ASAP and ACE were first started, especially ASAP was first started to get a lot of national attention. Many people would poke around on our website and say, oh, we already have three of these five things you’re doing. So we just need help on two of them. That’s not what the program is. The program is about the way it’s knitted together and the way it’s implemented. So I would encourage any higher ed leaders, folks in the legislature, anyone that’s interested in learning more about it, maybe just to read a couple of the studies just to understand the comprehensive model. And then if you’re still interested, which I hope they would be, is to then turn to the replication collaborative website to learn about the technical assistance that’s available.

0:28:09.5 HD: Yeah, any other resources that you would point them to, to learn more about ACE and ASAP or any of the programs?

0:28:17.5 DL: So, I mean, we’re just in the, I mean, our SUNY ACE and ASAP website is up and you can learn a little bit about which colleges are involved and how we’re going about the work within SUNY. I think if folks really wanna learn about the model, the CUNY website, which includes a rich array of videos, you can hear actual student voices. SUNY is gonna get there. We’re just not quite there yet because we’re at the beginning of our journey, but you wanna learn about the work we’re doing within SUNY. Certainly come to the SUNY website and just go directly to suny.edu and type in ASAP. It’ll take you right to our page. Same thing with CUNY. Go to the CUNY website and type in ACE or ASAP and it’ll take you right there where you can learn more about it.

0:29:01.5 HD: Fantastic. Donna Linderman, thank you so much for joining us today.

0:29:09.5 DL: Thank you, it was a pleasure, Heather.

[music]