Dropouts - A Badly-Reported Problem

I have been following school issues in Anne Arundel County since the mid-nineties, when my children were at West Annapolis Elementary.

For those who don't live in the Annapolis area, let me describe the school system briefly. School systems in Maryland are set up on a county-wide basis. Anne Arundel County's school system serves 75,000 children, grouped into twelve feeder systems. Here in Annapolis, we have nine elementary schools, two middle schools, and one high school.

While the overall population of the Annapolis area is seventy-five percent white (or was, in the 2000 census), there's been a modest black majority enrolled in public schools here. White students come overwhelmingly from middle class and upper middle class families. Black students, though, come from a more diverse background. Some are from upper middle class families, a fair number are from middle class families, and a sizeable number are also from low income families, many of whom live in public housing.

If I were doing the same kind of analysis on our public schools that a consulting firm would do inside a large corporation - think Verizon, or one of the other phone companies - I'd do a Leaky Pipe Analysis. How many kids make it all the way through to the end successfully? How many kids fall by the wayside as they go through, receiving extra tutoring perhaps, or extra discipline, or perhaps dropping out before graduation? How much "Leakage" do we have in the public schools?

The first time I looked at enrollment numbers, I was struck by the contrast between 9th grade enrollment counts and 12th grade enrollment counts. I did a few quick calculations, and discovered an average drop of twenty-seven percent between 9th grade and 12th grade. Not 27% just in Annapolis. A 27% drop county-wide.

Then Mary Alice Gehrdes - one of the most dedicated PTA moms in the world - pointed out that 9th grade isn't the right starting point. Too many high school students fail 9th grade and then repeat the grade. So I changed focus, and began comparing 12th grade enrollment counts with the 8th grade enrollment counts from four years earlier, to see what the county's loss rate looks like from an 8th Grade To 12th Grade perspective.

I began accumulating some interesting sets of numbers. And I also found myself wading into a territory that's not very pretty. Public schools, it turns out, make a point of keeping the public confused about how poorly they're doing.

Here's one of the first sets of numbers I saved.

I thought this was absolutely extraordinary. More than 300 African-American students in the freshman class. Including, of course, a number of holdovers from the previous year(s). Barely more than 100 African-American students in the senior class.

Meanwhile, the white enrollment was just over 200 for the freshman class, a little under 200 in the senior class.

If you looked at Annapolis High's freshman class, you saw a high school that's majority black.

If you looked at Annapolis High's senior class, you saw a high school that's majority white.

Then I discovered something even more curious. The reported county-wide dropout rate was less than five percent! Here's the data series that I dropped into my files several years ago.

Something about this smelled seriously wrong. How can Anne Arundel County have a senior class enrollment that's 27% lower than its freshman class enrollment from three years earlier, and yet report an annual dropout rate of 4.16%?

As it turns out, public school systems have two basic choices on how to calculate and report dropout rates, one that hides their performance problems from the public, and one that shares their performance shortcomings. The rate they use is the one that hides the truth. It's the School Year Rate (technically known as the Event Rate). This year's dropouts divided by this year's enrolled students equals this year's dropout rate.

Let me re-state that, though, so you'll have a better sense of what's going on. The School Year Rate divides one cohort's worth of dropouts by four cohorts of enrolled students. The School Year Rate automatically waters down the real dropout rate by seventy-five percent, if you live in Maryland.

If you live in DC, or in Virginia, the true rate is watered down even more. In those jurisdictions, at the time I first researched this issue, local school systems added 7th and 8th grade enrollment counts to the denominator as well. They divided one cohort's worth of dropouts by six years of enrolled students! As a result, their reported dropout rates looked very small.

The honorable and truthful way to calculate and report dropouts is on a Graduating Class basis, or, more technically, on a Cohort basis. (A "cohort" is a group of kids all born at the same time who all go through the school system together and all graduate at about the same time.)

In a Cohort system, all the dropouts from the Class of 2006, say, would be added together for every year from 9th grade on. And the school system would divide that count by the total number of 9th graders in the fall of 2002, when the Class of 2006 first entered high school.

Enough of the technical stuff for the moment. Let's look at a few more graphs. The following seven charts show the year by year enrollment numbers for the graduating classes of 1993 through 1999. In each chart, the first bar represents county-wide enrollment in the class's Kindergarten year. The last bar represents countywide enrollment in its Senior year. The percent rate highlighted in the bottom right corner represents the enrollment drop from 8th grade to 12th grade.

An 8th grade enrollment of 4,528 had declined to a 12th grade enrollment of 3,699. (All counts are based on measurements taken at the end of September, just as the school year really gets rolling.)

Same pattern. A 17% enrollment decline.

Here a 20% enrollment decline.

Now a 25% enrollment decline.

And a 23% enrollment decline. Think, for a minute, about who it is that disappears. Do the children of upper middle class families drop out at such a high rate? Of course not. Most of the dropouts come from what is rudely called "the bottom third." The dropout rate is much higher than 23% for kids from low income backgrounds. And much lower than 23% for kids from upper income families.

Here the enrollment loss is up to 26%.

Now that you have a feel for the patterns, let's look again at how the school system typically reports its dropout rates. (At the time I did this chart, one number wasn't available - and I haven't yet gotten around to filling it in.)

In the 1992 - 93 school year, Anne Arundel County Public Schools reported a dropout rate of 3.76%. Yet the Class of '93 was 18% smaller than it had been four years earlier, in 8th grade. And so on, through the following six cohorts, the following six graduating classes.

One more anecdote. When I first gathered the data from which these charts were built, I wanted to add one final bar on the right. I wanted the spring graduation count for each of these classes. I called the school system, and was referred to the data keeper who's most on top of these numbers. I asked her if she could supply me with the spring graduation numbers.

No, I can't, she told me. We don't keep track of the spring graduation count. I've been in this job for three years. The person before me didn't track the spring graduation count. In the three years I've been here, no one has asked me for the spring graduation count. So I don't keep it. The only number we keep track of is the "completer" count, which runs from July 1 to the following June 30.

In other words, the "completer" count starts with kids who earn their final graduation credits in summer school, or in the following fall semester, and ends with the graduate count from the next spring's commencement ceremonies. It's not a clean count of all the kids who graduate from a single class, it's a count that combines students from two or more classes.

This struck me as extraordinary. Anne Arundel County taxpayers provide hundreds of millions of dollars a year to the school system. But the school system doesn't even have enough pride in what it does to count its own graduates each spring, publish the number for the newspapers, and keep track of the numbers so it can see whether it's making progress or not.

A National Scandal, Signs of Progress

Not long after I'd discovered just how it is a school system can report a dropout rate of four percent plus even though nearly a fifth of its students disappear before graduation, I happened to be chatting with a friend who had just begun to do some consulting work for the National Center on Education Statistics (NCES), inside the Department of Education. I explained the flaws in the School Year Dropout Rate. I described the virtues of the Cohort Dropout Rate, i.e., the Graduating Class Dropout Rate. I suggested he encourage the NCES to favor the Cohort Rate, and he nodded.

Some months later, we met again. He had suggested the Cohort Rate to the NCES, he told me. And they nearly bit his head off. "We would NEVER do that!" he was told, and he felt he'd probably be out the door if he raised the issue again.

Local superintendents don't like the idea of letting the public know just how bad the dropout rate is. State Superintendents don't care for the idea either. And the National Center for Education Statistics plays along with them.

This constant foot-dragging is now being challenged. The National Governor's Association takes the dropout issue quite seriously. In a recent meeting, the Governor's endorsed model legislation that would require school districts to calculate and report their graduation rates on a cohort basis. If adopted, every school district would have to play by the same rules. Total graduates divided by total new 9th graders four years later equals the graduation rate (with adjustments for Transfers Out and Transfers In).

In its 2006 session, the Maryland legislature passed a bill that requires all districts in the state to begin reporting graduation rates on a cohort basis by 2010. (Thanks go to Delegates Gutierrez and Marriott and Senator Britt for initiating and sponsoring this legislation.)

The Anne Arundel County Board has now gone the legislature one better. In a recent resolution, it authorized the Superintendent to calculate and report the county dropout rate on a cohort basis as well. The proof of the pudding is in the implementation, and as of this writing, the cohort method for calculating dropout rates has yet to be implemented. In time, though, the public ought to start receiving a much more accurate statement of the dropout problem.

How's Annapolis High doing?

In response to a data request, the school system recently provided numbers showing total African-American graduates, and total African-American dropouts, at Annapolis High. The problem is still quite real. Too many African-American students still don't make it to the finish line of Commencement. But the numbers don't appear to be quite as bad as they were a decade ago. Some progress has been made. Much more is still needed.

This chart is quite basic. It doesn't line anything up by cohort. Of the 69 students recorded as dropping out during the 2001-02 school year, many would have been second year freshmen, some would have been sophomore, some juniors, and some seniors. The same is true for each of the succeeding school years. If the numbers are accurate - and that's always iffy - the dropout loss rate at Annapolis High isn't quite as bad as it was a decade ago.

What does this mean? Does this mean that children from low income neighborhoods are doing better in school? One hopes so, though the school system doesn't actually track graduation rates by neighborhood. (It could, and it should, but it doesn't).

Might it mean that families who live in low income neighborhoods are aging in place, and not sending as many children into the public schools as they used to? I have no idea.

What remains of all this, I think, are a couple of sad but important lessons. School systems in general, and the Anne Arundel system in particular, are not in the habit of telliing themselves the hard truths about how poorly they perform in educating children of lower income families and neighborhoods.

On the other hand, until school systems face up to their shortcomings, it's unlikely that they'll muster the energy to change.

I'll have more to say on this point in a later page.

Steve Johnson
The Wallcharts Workshop
April 19, 2006