In October of 2017, the venerable James Ford of the Public School Forum delivered the keynote address at the North Carolina English Teacher’s Association. It was more than exceptional as Ford highlighted that what hurts our schools most are external factors that are not being dealt with such as systemic poverty.

Part of his presentation included a version of what is called the “Iceberg Effect” for education. It looks like this:

Ford talked about (and he is not alone in this belief) how what is above the water, namely student outcomes, is what drives educational policies in our state.

Notice that he means what is visible above the water line is what drives policy. That is what the public sees in the press. That is what lawmakers and leaders hark on when discussing what to do about public education. That is what is being used to measure the effectiveness of teachers and schools.

In 2013, the state of North Carolina started using a value-added measurement scale to help gauge teacher effectiveness and school performance. Developed by SAS which is headquartered in the Triangle area of Raleigh, Durham, and Chapel Hill, EVAAS collects student data and creates reports that are used to measure teacher and school effectiveness.

EVAAS stands for “Education Value-Added Assessment System.” For teachers, it is supposed to give an indication of how well students are supposed to do in a given year on the tests that are used on evaluations. (Do not let it be lost on anyone that “EVAAS” scores were just released at the end of most schools’ first quarter after half of the block classes have already completed more than half of the curriculum’s work).

EVAAS has been the subject of a lot of scrutiny. It deserves every bit of that scrutiny. Why? Because the algorithms that it uses to come up with its calculations and reports are like a tightly held secret.

Think of the iceberg and what is seen and what is under the water line.

The North Carolina Department of Public Instruction describes EVAAS as:

EVAAS examines the impact of teachers, schools, and districts on the learning of their students in specific courses, grades, and subjects. Users can access colorful, easy-to-understand charts and graphs via the Web, as well as produce customized reports that predict student success, show the effects of schooling at particular schools, or reveal patterns in subgroup performance (http://www.ncpublicschools.org/effectiveness-model/evaas/).

There is even a nice little video that one can go to in order to “understand how EVAAS” works (https://ncdpi.sas.com/videos/EVAAS/WhatIsEVAAS.mp4).

The whole video is an attempt to validate the use of EVAAS by the state. Except it does not tell anyone how “EVAAS performs value-added analysis.” The only people who know how that works are inside of the Hawkins National Laboratory or as we know it, SAS headquarters.

In March of 2017, Angela Scioli wrote a powerful piece for EDNC.org entitled “EVAAS: An incomplete and painful system for me.” In it she stated,

I did not change anything else about my teaching.  I did not know what to change.  No one met with me to intervene.  No one even spoke to me about the results.  It just sat there, like a black eye I couldn’t cover up, but no one wanted to talk about it.  

The next year, I received my EVAAS results, after using the same methods, and I was now deemed “highly effective.”  I was relieved and confused.  How could that be? (https://www.ednc.org/2017/03/21/evaas-incomplete-painful-system/).   

Justin Parmenter’s op-ed entitled “The cost of doing business in the education world” (August 9, 2017) was another powerful expose of a world in which EVAAS is being used to measure teachers and schools. He said,

In the years that followed, EVAAS was rolled out on a larger scale across the district and state, and similar data measuring teacher effectiveness was made available to more teachers. I was dismayed to see that, while some years I apparently had made a difference, there were other years when I did not make much of a difference at all. Some years I even made a negative difference (https://www.ednc.org/2017/08/09/cost-business-education-world/).

This criticism of EVAAS is not limited to North Carolina. From the National Education Policy Center:

“Education Policy Analysis Archives recently published an article by Audrey Amrein-Beardsley and Clarin Collins that effectively exposes the Houston Independent School District use of a value-added teacher evaluation system as a disaster. The Educational Value-Added Assessment System (EVAAS) is alleged by its creators, the European software giant SAS, to be the “the most robust and reliable” system of teacher evaluation ever invented. Amrein-Beardsley and Collins demonstrate to the contrary that EVAAS is a psychometric bad joke and a nightmare to teachers” (http://nepc.colorado.edu/blog/houston-you-have-problem).

And the ambiguity of how SAS uses data within the EVAAS program is not lost on many people. From a 2014 WUNC report called “Ranking Teachers: NC Bets Big On A Complicated Stats Model,”

EVAAS is based on that student growth, not the test score itself. And the software is complicated – and some say largely secret. Teachers, principals, even administrators at the state level don’t know everything that goes into the model.

“Now the statisticians, and I’m not a statistician – I’m not the smartest guy in the world – they would say that stuff should even out, and I think they are correct, I’m sure it does even out, when you look at statewide data,” says Jim Key, an assistant superintendent in Durham. “But within a particular classroom? You could have more than a normal share of students who are going through some challenges with their personal lives” (http://wunc.org/post/ranking-teachers-nc-bets-big-complicated-stats-model#stream/0).

That last quote from Mr. Key accurately sums up the relationship between the EVAAS program and the Iceberg Effect.

Simply put, EVAAS only measures what is the tip of the iceberg that is above the water and then it tells us how to view it. It completely disregards what is under the water level.

LET ME REPEAT: EVAAS ONLY MEASURES WHAT IS TO BE SEEN, NOT WHAT LIES UNDERNEATH. IT IS ONLY CONCERNED WITH STUDENT OUTCOMES. IT DOES NOT ACCOUNT FOR “INEQUITY & INEQUALITY,” “STRESS & VIOLENCE,” “SUPPORT FOR SCHOOLS,” AND “SUPPORT FOR YOUNG FAMILIES.”

Teachers and schools measured by EVAAS actually have to battle against all of the iceberg, not just the tip which is by far the smallest part of the iceberg.

The state pays more than three million dollars annually to SAS which was co-founded and is still run by Jim Goodnight who according to Forbes Magazine is one of the top donating executives to political campaigns. In 2016 he donated much to a PAC for Jeb Bush who while in Florida instituted the school performance grade system that North Carolina uses now – the same one that utilizes EVAAS reports to measure schools (https://www.bizjournals.com/triangle/blog/techflash/2015/10/forbes-sas-goodnight-among-tech-execs-for-top.html).

It also is worth looking at the fact that his wife, Ann Goodnight, is a co-founder and board member of BEST NC. When BEST NC had its 2018 legislative meeting it brought in the toxic Michelle Rhee and her campaign for value-added measurements to discuss policy. That “closed-door” meeting was held at SAS headquarters.

The recent principal pay schedule that has garnered well-deserved criticism was spearheaded by BEST NC with legislators behind the scenes over the summer utilizes EVAAS data.

Too much is being dictated by a private entity that is privately calculating data in a secret fashion to measure a public good and how much should be spent on that public good in a state that wants to privatize that public good.

There’s too much incestuous synergy there. And all of it is purposefully ignoring the part of the iceberg that is beneath the water line.

In using EVAAS, what the state of North Carolina is doing is sending schools on expeditions in remote icy waters without the use of radar and sonar to navigate themselves. It’s like the Titanic.

But instead of being surprised at the fact that the “unsinkable” actually succumbed to a lonely iceberg, the state has already made a hole in the hull for water to leak in, so even if the “ships” avoid hitting icebergs, they would already have a hard time reaching port. As the “unsinkable” ships begin to sink, the state says we must invest in other alternatives like charter school reform and vouchers, so the money starts going to other modes of “transportation.”

The problem is that the icebergs in our state are getting bigger and more are breaking off. As the income gap widens and as segregationist tendencies begin to take firmer root, systems like EVAAS will still serve as a façade of the actual truth which lies beneath the water.

Of course, SAS could release how it uses data and calculates its reports but that would require transparency.

But icebergs work best in cold, murky, choppy waters. And people in Raleigh like having big icebergs.