In October of 2017, the venerable James Ford of the Public School Forum delivered the keynote address at the North Carolina English Teacher’s Association. It was more than exceptional as Ford highlighted that what hurts our schools most are external factors that are not being dealt with such as systemic poverty.

Part of his presentation included a version of what is called the “Iceberg Effect” for education. It looks like this:

Ford talked about (and he is not alone in this belief) how what is above the water, namely student outcomes, is what drives educational policies in our state.

Notice that he means what is visible above the water line is what drives policy. That is what the public sees in the press. That is what lawmakers and leaders hark on when discussing what to do about public education. That is what is being used to measure the effectiveness of teachers and schools.

In 2013, the state of North Carolina started using a value-added measurement scale to help gauge teacher effectiveness and school performance. Developed by SAS which is headquartered in the Triangle area of Raleigh, Durham, and Chapel Hill, EVAAS collects student data and creates reports that are used to measure teacher and school effectiveness.

EVAAS stands for “Education Value-Added Assessment System.” For teachers, it is supposed to give an indication of how well students are supposed to do in a given year on the tests that are used on evaluations. (Do not let it be lost on anyone that “EVAAS” scores were just released at the end of most schools’ first quarter after half of the block classes have already completed more than half of the curriculum’s work).

EVAAS has been the subject of a lot of scrutiny. It deserves every bit of that scrutiny. Why? Because the algorithms that it uses to come up with its calculations and reports are like a tightly held secret.

Think of the iceberg and what is seen and what is under the water line.

At the time the North Carolina Department of Public Instruction described EVAAS as:

EVAAS examines the impact of teachers, schools, and districts on the learning of their students in specific courses, grades, and subjects. Users can access colorful, easy-to-understand charts and graphs via the Web, as well as produce customized reports that predict student success, show the effects of schooling at particular schools, or reveal patterns in subgroup performance.

There was even a nice little video that one can go to in order to “understand how EVAAS” works (https://ncdpi.sas.com/videos/EVAAS/WhatIsEVAAS.mp4).

The whole video is an attempt to validate the use of EVAAS by the state. Except it does not tell anyone how “EVAAS performs value-added analysis.” The only people who know how that works are inside of the Hawkins National Laboratory or as we know it, SAS headquarters.

The ambiguity of how SAS uses data within the EVAAS program is not lost on many people. From a 2014 WUNC report called “Ranking Teachers: NC Bets Big On A Complicated Stats Model,”

EVAAS is based on that student growth, not the test score itself. And the software is complicated – and some say largely secret. Teachers, principals, even administrators at the state level don’t know everything that goes into the model.

“Now the statisticians, and I’m not a statistician – I’m not the smartest guy in the world – they would say that stuff should even out, and I think they are correct, I’m sure it does even out, when you look at statewide data,” says Jim Key, an assistant superintendent in Durham. “But within a particular classroom? You could have more than a normal share of students who are going through some challenges with their personal lives” (http://wunc.org/post/ranking-teachers-nc-bets-big-complicated-stats-model#stream/0).

That last quote from Mr. Key accurately sums up the relationship between the EVAAS program and the Iceberg Effect.

Simply put, EVAAS only measures what is the tip of the iceberg that is above the water and then it tells us how to view it. It completely disregards what is under the water level.

LET ME REPEAT: EVAAS ONLY MEASURES WHAT IS TO BE SEEN, NOT WHAT LIES UNDERNEATH. IT IS ONLY CONCERNED WITH STUDENT OUTCOMES. IT DOES NOT ACCOUNT FOR “INEQUITY & INEQUALITY,” “STRESS & VIOLENCE,” “SUPPORT FOR SCHOOLS,” AND “SUPPORT FOR YOUNG FAMILIES.”

Teachers and schools measured by EVAAS actually have to battle against all of the iceberg, not just the tip which is by far the smallest part of the iceberg.

The state still pays more than four million dollars annually to SAS which was co-founded and is still run by Jim Goodnight who according to Forbes Magazine is one of the top donating executives to political campaigns. In 2016 he donated much to a PAC for Jeb Bush who while in Florida instituted the school performance grade system that North Carolina uses now – the same one that utilizes EVAAS reports to measure schools (https://www.bizjournals.com/triangle/blog/techflash/2015/10/forbes-sas-goodnight-among-tech-execs-for-top.html).

Too much is being dictated by a private entity that is privately calculating data in a secret fashion to measure a public good and how much should be spent on that public good in a state that wants to privatize that public good. In using EVAAS, what the state of North Carolina is doing is sending schools on expeditions in remote icy waters without the use of radar and sonar to navigate themselves. It’s like the Titanic.

But instead of being surprised at the fact that the “unsinkable” actually succumbed to a lonely iceberg, the state has already made a hole in the hull for water to leak in, so even if the “ships” avoid hitting icebergs, they would already have a hard time reaching port. As the “unsinkable” ships begin to sink, the state says we must invest in other alternatives like charter school reform and vouchers, so the money starts going to other modes of “transportation.”

The problem is that the icebergs in our state are getting bigger and more are breaking off. As the income gap widens and as segregationist tendencies begin to take firmer root, systems like EVAAS will still serve as a façade of the actual truth which lies beneath the water.

Of course, SAS could release how it uses data and calculates its reports but that would require transparency.

But icebergs work best in cold, murky, choppy waters. And people in the NC General Assembly like having big icebergs.