When .gov Allows .edu To Be Governed By .com – North Carolina’s Allegiance to EVAAS

At the beginning of each school year, I am required to fully disclose my syllabus to all perspective students and parents.

On the first day of class, I give each student a set of rubrics that I use to gauge written work throughout the year.

Any student can ask how any assessment was graded and conference about it.

That’s part of my job.

Does the state do that for each school when school performance grades and school report cards are published?

Last month, this blog published a post on the opaque relationship that our state has with SAS and and its EVAAS value-added measurement tools – https://caffeinatedrage.com/2017/11/26/why-teachers-should-be-wary-of-evaas-and-sas/.

And here is another item to consider.

Last week, State Superintendent of Public Schools Mark Johnson released a video to all public school teachers announcing the new revamped state school report card system.

Here is a frame that is closed captioned –

src1

It says, “Recently, I launched the brand-new website for school report cards: schoolreportcards.nc.gov.”

That means it should be controlled by the state, correct?

Put that into your search bar and you get:

src2

It’s not the actual report card site – just a “Welcome” page. Notice that it has a link to the actual school report card site along with the following text:

The School Report Card website has been completely redesigned for 2017. This interactive website, designed and hosted by SAS, includes printable versions of the North Carolina School Report Card snapshots. For researchers and others who want to dig into the data further, an analytical site is available here.

The actual “School Report Card” website has a different domain name.

src3

It’s https://ncreportcards.ondemand.sas.com/src.

Actually, the chain is from a .gov to a .org to a .com.

There is a link “for researchers and others who want to dig into the data further – an analytical site.”

There is a lot to explore in the analytical site, but where is the actual rubric, the formula for calculations, the explanation of how achievement and growth come together to get this report card?

If a teacher could not explain exactly how a grade was calculated, then that teacher’s assessment would be called into doubt.

Except here, we have an entire state spending taxpayer money to a company that will not publish its “rubric” and “calculations” for its own assessment.

 

 

 

The New North Carolina State Report Cards And What They Really Show

“The transformation of our public education system will open true pathways out of poverty.”

 – Mark Johnson, September 7th, 2016 from an op-ed entitled “Our American Dream” (https://www.ednc.org/2016/09/07/our-american-dream/).

 

This week State Superintendent of Public Schools Mark Johnson presented a new school report card interface and “updated features” so that the public can view school report cards (https://ncreportcards.ondemand.sas.com/src/index). It has a lot of bells and whistles.

The letter attached to its new release by Johnson seems well-meaning. The text can be found here – http://www.ncpublicschools.org/src/welcome/.

Yet, no matter how much glitter and glam can be used to create an interface that appeals to the eyes, it doesn’t cover up the fact that there really is so much more that makes up a school than a school report card in this state chooses to measure.

Yes, Johnson does make note in his letter that there is more to a school than a “grade.” He states,

“As a former teacher, I can tell you this information, while important, cannot tell you the entire story of a school. These facts and figures cannot voice the extra hours put in by your teachers preparing for class and grading assignments, the school spirit felt by families, the involvement in sports, arts, or other extracurriculars that build character, and other crucial aspects of a school community.”

But the school report cards still do not reflect those very considerations that give a school so much of its identity and define its true outreach to the students and the communities they serve. In fact, that is one of the many glaring items deficiencies that come to mind when reviewing the new interface.

  1. It totally ignores the fact that what affects so many schools is POVERTY.

As soon as one accesses the site, a map of the state is shown.

Picture1

One can then drill down from there. But one has to wonder if there is any measurement of certain socio-economic trends besides the number of kids on free and reduced lunches.

What about the effects of the gerrymandering that has occurred in recent years in the drawing of districts? What about how the unconstitutional VOTER ID law affected how people could vote and put representatives in Raleigh who would fight more for their students?

EdNC.org has a useful tool on its site called the Data Dashboard. You can find it here – https://www.ednc.org/data/.  Take the time to peruse this resource if public education is a top issue for you.

Here is a dot map of the 2014-2015 school performance grade map for the state (https://www.ednc.org/2015/08/03/consider-it-mapped-and-school-grades/) .

Picture2

Take notice of the pink and burgundy dots. Those are schools in the “D” and “F” category.

Now look at a map from the dashboard for Free and Reduced lunch eligibility for the same year.

Picture3

If you could somehow superimpose those two images, you might some frighteningly congruent correlations.

What if that capability was allowed within the new interface of the school report cards?

Now take a look again at the quote from Mark Johnson at the beginning of this posting:

“The transformation of our public education system will open true pathways out of poverty.”

I would argue that addressing poverty outside of class would help students inside of class as much if not more. Besides showing people how many textbooks there are per student (which is probably not correct as school systems are constantly shuffling textbooks around to cover the needs), what about the per capita measurements?

Education can help pull people out of poverty. I will not argue that, but attacking poverty at its root sources will do so much to help education. This revamped site seems to totally ignore that.

And maybe Johnson’s revamped school report site should also include this graph.

Picture4

That is from the 2015–16 Performance and Growth of North Carolina Public Schools Executive Summary, NC DPI.

  1. This site is being used as a way to promote more privatization through the veiled crusade of SCHOOL CHOICE.

Mark Johnson is about “school choice.” He has said so.

Those school performance grades that appear so quickly when one drills down on a district are based on a model developed by Jeb Bush when he was in Florida. It’s disastrous and places a lot of emphasis of achievement scores of amorphous, one-time testing rather than student growth throughout the entire year.

It’s part of the “proficiency versus growth” debate that really came to the forefront during the Betsy DeVos confirmation hearings when she could not delineate between whether test scores are used to measure student “achievement” or student “growth.”

Consider this:

Picture5

Interestingly enough, in the school year 2019-2020, the school performance grade scale will shift from a fifteen-point scale to a ten-point scale. Do you know what that means?

IT WILL BE HARDER FOR PUBLIC SCHOOLS TO QUALIFY AS PASSING. IN FACT, SCHOOLS COULD HAVE A HIGHER PERCENTAGE OF STUDENT GROWTH AND STILL GET A LOWER SCHOOL PERFORMANCE GRADE! AND THE SCHOOL REPORT CARD SITE WILL HIGHLIGHT THAT!

There will be more failing schools. This comes from a legislative body that endorsed the state board last school year to institute a ten-point scale for all high school grading systems to help ensure higher graduation rates, but now shrinks scales for those schools’ performance grades.

This comes from the same legislative body that literally is propping up the very state superintendent who is championing this very site.

Guess what else is happening in 2019-2010? Voucher expansion! From the recent session that gave us our current budget:

SECTION 6.6.(b) G.S. 115C-562.8(b) reads as rewritten: “(b) The General Assembly finds that, due to the critical need in this State to provide opportunity for school choice for North Carolina students, it is imperative that the State provide an increase of funds of at least ten million dollars ($10,000,000) each fiscal year for 10 years to the Opportunity Scholarship Grant Fund Reserve. Therefore, there is appropriated from the General Fund to the Reserve the following amounts for each fiscal year to be used for the purposes set forth in this section: 


Fiscal Year Appropriation

2017-2018 $44,840,000
2018-2019 $54,840,000
2019-2020 $64,840,000
2020-2021 $74,840,000
2021-2022 $84,840,000
2022-2023 $94,840,000
2023-2024 $104,840,000
2024-2025 $114,840,000
2025-2026 $124,840,000
2026-2027 $134,840,000

Bottom line is that this site is helping to fuel the slanted and loaded argument that what this state needs more of is SCHOOL CHOICE! However, what is happening in this state is that “school choice” really is a euphemism for unregulated charter schools and vouchers – neither of which have produced results that show improvement for student achievement.

  1. The site is maintained by SAS.

Look at the web address – https://ncreportcards.ondemand.sas.com/src/. That “sas” represents SAS, the same SAS that controls EVAAS which measures schools by a secret algorithm. That “.com” means it’s maintained by a commercial entity. It gets paid taxpayer money.

Back to Johnson’s letter accompanying the new website:

“We launched the new website, a completely redesigned online resource that provides the transparency you need into the characteristics and performance of your school in an easy-to-use format, to better inform you. I encourage you to follow the link to a school’s individual website to find out more about the school’s full story.”

There’s a word there called “transparency.” EVAAS is the very epitome of not being transparent.

Actually, it is rather mindboggling to think that a measurement which comes from EVAAS is so shrouded in so much opaqueness. With the power to sway school report cards and school performance grades, it would make sense that there be so much transparency in how it calculates its data so that all parties involved would have the ability to act on whatever needs more attention.

And people are literally invited to take action on the data presented by the school report card website. In fact, SAS’s measurement slaps you in the face as soon as you choose a district or school.

Picture6

In fact, if one chooses to look at a district, then all schools are displayed according by color to whether they met growth and with a large letter grade. It’s like they are already being compared against each other when the very makeup of the schools and the obstacles each faces could differ a lot.

Think about what a school report card might not show.

  1. Does the school report card show how successful graduates are in post-secondary educational endeavors like Virginia which has dropped the performance grading system?
  2. Does the school report card consider the viewpoints of the parents whose students are being taught? school report card
  3. Does the school report card consider the viewpoints of the students and how they feel about the learning experience and their security in the school and the classroom?
  4. Does the school report card consider how many students are taking “rigorous” courses?
  5. Does the school report card consider the amount of community service done by students in the school?
  6. Does the school report card consider the strength of the drama department and the quality of the productions?
  7. Does the school report card consider what is seen in the yearbook?
  8. Does the school report card consider the strength of the student newspaper?
  9. Does the school report card consider the strength of the JROTC program?
  10. Does the school report card consider the number of viable clubs and organizations on campus?
  11. Does the school report card consider the amount of scholarship money won by graduating students?
  12. Does the school report card consider the number of student participating in sports?
  13. Does the school report card consider the number of foreign languages offered?
  14. Does the school report card consider the number of students in the Student Section at a game?
  15. Does the school report card consider the number of students who wear spirit wear?
  16. Does the school report card consider the number of students involved in choral and musical endeavors?
  17. Does the school report card consider the number of students who attend summer academic study opportunities?
  18. Does the school report card consider the quality of the artistic endeavors of students through visual and performance arts programs?
  19. Does the school report card consider the strength of programs that hope to help marginalized students?
  20. Does the school report card consider the transient rate of the student body?
  21. Does the school report card consider the poverty levels of the surrounding area that the school services?
  22. Does the school report card consider the number of students who hold jobs?
  23. Does the school report card consider the effect of natural disasters such as hurricanes?
  24. Does the school report card consider the funding levels of the programs?
  25. Does the school report card consider the number of students on 504 plans or IEP’s?
  26. Does the school report card consider the rations of nurses and counselors to students?
  27. Does the school report card consider the class sizes?

Yes, this new interface for the school report cards of NC’s public schools looks modern and it does show data in a more eye-friendly manner, but what it really displays is how unwilling this current crop of policy makers are in confronting what really affects our schools, especially poverty.

It also is proof that Mark Johnson is more interested in the appearance of doing well.

And appearances are deceiving.

 

The Malleable, Nebulous, and Secretive Algorithm of EVAAS

Recently this blog released a post concerning SAS and its value-added measurement software called EVAAS detailing its rather “undetailed” functionality at least as it deals with North Carolina (https://caffeinatedrage.com/2017/11/26/why-teachers-should-be-wary-of-evaas-and-sas/).

evaas header.PNG

Be reminded the North Carolina Department of Public Instruction describes EVAAS as:

EVAAS examines the impact of teachers, schools, and districts on the learning of their students in specific courses, grades, and subjects. Users can access colorful, easy-to-understand charts and graphs via the Web, as well as produce customized reports that predict student success, show the effects of schooling at particular schools, or reveal patterns in subgroup performance (http://www.ncpublicschools.org/effectiveness-model/evaas/).

It is rather mind-boggling to think that a measurement which comes from EVAAS is shrouded in so much opaqueness. With the power to sway school report cards and school performance grades, it would make sense that there be so much more transparency in how it calculates its data so that all parties involved would have the ability to act on whatever needs more attention.

But for right now, we are relegated to asking questions about how SAS uses EVAAS to come up with its measurements.

So when a teacher receives an EVAAS score can a person from DPI or even SAS answer the following questions when it comes to “measuring performance” that would truly allow that teacher to be convinced his / her effectiveness has been measured correctly?

  1. Do EVAAS scores consider the viewpoints of the parents whose students are being taught?
  2. Do EVAAS scores consider the viewpoints of the students and how they feel about the learning experience and their security in the school and the classroom?
  3. Do EVAAS scores consider the growth index of a school?
  4. Do EVAAS scores consider the attendance records of the students?
  5. Do EVAAS scores consider the graduation rate?
  6. Do EVAAS scores consider how many students are taking “rigorous” courses?
  7. Do EVAAS scores consider the amount of community service done by students in the school?
  8. Do EVAAS scores consider the strength of the drama department and the quality of the productions?
  9. Do EVAAS scores consider what is seen in the yearbook?
  10. Do EVAAS scores consider the strength of the student newspaper?
  11. Do EVAAS scores consider the strength of the JROTC program?
  12. Do EVAAS scores consider the number of viable clubs and organizations on campus?
  13. Do EVAAS scores consider the amount of scholarship money won by graduating students
  14. Do EVAAS scores consider the number of student participating in sports?
  15. Do EVAAS scores consider the number of teachers who have advanced degrees and advanced certification?
  16. Do EVAAS scores consider the number of foreign languages offered?
  17. Do EVAAS scores consider the number of students in the Student Section at a game?
  18. Do EVAAS scores consider the number of students who wear spirit wear?
  19. Do EVAAS scores consider the number of students involved in choral and musical endeavors?
  20. Do EVAAS scores consider the number of students who attend summer academic study opportunities?
  21. Do EVAAS scores consider the quality of the artistic endeavors of students through visual and performance arts programs?
  22. Do EVAAS scores consider the strength of programs that hope to help marginalized students?
  23. Do EVAAS scores consider the transient rate of the student body?
  24. Do EVAAS scores consider the turnover rate of the staff and faculty?
  25. Do EVAAS scores consider the poverty levels of the surrounding area that the school services?
  26. Do EVAAS scores consider the number of students who hold jobs?
  27. Do EVAAS scores consider the effect of natural disasters such as hurricanes?
  28. Do EVAAS scores consider the funding levels of the programs?
  29. Do EVAAS scores consider the number of students on 504 plans or IEP’s?
  30. Do EVAAS scores consider the rations of nurses and counselors to students?
  31. Do EVAAS scores consider the class sizes?

And that’s not even really directly touching on issues related to poverty.

If any or all of those are considered in calculating EVAAS scores then SAS should prove it.

If any or all of those are NOT considered, then maybe SAS should take a deeper look at whether or not they should be considered.

Because all of them has an effect on students and schools in varying ways.

Quantify that.

 

Why Teachers Should Be Wary of EVAAS and SAS

In October, the venerable James Ford of the Public School Forum delivered the keynote address at the North Carolina English Teacher’s Association. It was more than exceptional as Ford highlighted that what hurts our schools most are external factors that are not being dealt with such as systemic poverty.

Part of his presentation included a version of what is called the “Iceberg Effect” for education. It looks like this:

iceberg

Ford talked about (and he is not alone in this belief) how what is above the water, namely student outcomes, is what drives educational policies in our state.

Notice that he means what is visible above the water line is what drives policy. That is what the public sees in the press. That is what lawmakers and leaders hark on when discussing what to do about public education. That is what is being used to measure the effectiveness of teachers and schools.

In 2013, the state of North Carolina started using a value-added measurement scale to help gauge teacher effectiveness and school performance. Developed by SAS which is headquartered in the Triangle area of Raleigh, Durham, and Chapel Hill, EVAAS collects student data and creates reports that are used to measure teacher and school effectiveness.

EVAAS stands for “Education Value-Added Assessment System.” For teachers, it is supposed to give an indication of how well students are supposed to do in a given year on the tests that are used on evaluations. (Do not let it be lost on anyone that “EVAAS” scores were just released at the end of most schools’ first quarter after half of the block classes have already completed more than half of the curriculum’s work).

EVAAS has been the subject of a lot of scrutiny. It deserves every bit of that scrutiny. Why? Because the algorithms that it uses to come up with its calculations and reports are like a tightly held secret.

 

Think of the iceberg and what is seen and what is under the water line.

The North Carolina Department of Public Instruction describes EVAAS as:

EVAAS examines the impact of teachers, schools, and districts on the learning of their students in specific courses, grades, and subjects. Users can access colorful, easy-to-understand charts and graphs via the Web, as well as produce customized reports that predict student success, show the effects of schooling at particular schools, or reveal patterns in subgroup performance (http://www.ncpublicschools.org/effectiveness-model/evaas/).

There is even a nice little video that one can go to in order to “understand how EVAAS” works (https://ncdpi.sas.com/videos/EVAAS/WhatIsEVAAS.mp4).

EVAAS pic

The whole video is an attempt to validate the use of EVAAS by the state. Except it does not tell anyone how “EVAAS performs value-added analysis.” The only people who know how that works are inside of the Hawkins National Laboratory or as we know it, SAS headquarters.

This past March, Angela Scioli wrote a powerful piece for EDNC.org entitled “EVAAS: An incomplete and painful system for me.” In it she stated,

I did not change anything else about my teaching.  I did not know what to change.  No one met with me to intervene.  No one even spoke to me about the results.  It just sat there, like a black eye I couldn’t cover up, but no one wanted to talk about it.  

The next year, I received my EVAAS results, after using the same methods, and I was now deemed “highly effective.”  I was relieved and confused.  How could that be? (https://www.ednc.org/2017/03/21/evaas-incomplete-painful-system/).   

Justin Parmenter’s op-ed entitled “The cost of doing business in the education world” (August 9, 2017) was another powerful expose of a world in which EVAAS is being used to measure teachers and schools. He said,

In the years that followed, EVAAS was rolled out on a larger scale across the district and state, and similar data measuring teacher effectiveness was made available to more teachers. I was dismayed to see that, while some years I apparently had made a difference, there were other years when I did not make much of a difference at all. Some years I even made a negative difference (https://www.ednc.org/2017/08/09/cost-business-education-world/).

This criticism of EVASS is not limited to North Carolina. From the National Education Policy Center:

Education Policy Analysis Archives recently published an article by Audrey Amrein-Beardsley and Clarin Collins that effectively exposes the Houston Independent School District use of a value-added teacher evaluation system as a disaster. The Educational Value-Added Assessment System (EVAAS) is alleged by its creators, the European software giant SAS, to be the “the most robust and reliable” system of teacher evaluation ever invented. Amrein-Beardsley and Collins demonstrate to the contrary that EVAAS is a psychometric bad joke and a nightmare to teachers” (http://nepc.colorado.edu/blog/houston-you-have-problem).

And the ambiguity of how SAS uses data within the EVAAS program is not lost on many people. From a 2014 WUNC report called “Ranking Teachers: NC Bets Big On A Complicated Stats Model,”

EVAAS is based on that student growth, not the test score itself. And the software is complicated – and some say largely secret. Teachers, principals, even administrators at the state level don’t know everything that goes into the model.

“Now the statisticians, and I’m not a statistician – I’m not the smartest guy in the world – they would say that stuff should even out, and I think they are correct, I’m sure it does even out, when you look at statewide data,” says Jim Key, an assistant superintendent in Durham. “But within a particular classroom? You could have more than a normal share of students who are going through some challenges with their personal lives” (http://wunc.org/post/ranking-teachers-nc-bets-big-complicated-stats-model#stream/0).

That last quote from Mr. Key accurately sums up the relationship between the EVAAS program and the Iceberg Effect.

Simply put, EVAAS only measures what is the tip of the iceberg that is above the water and then it tells us how to view it. It completely disregards what is under the water level.

LET ME REPEAT: EVAAS ONLY MEASURES WHAT IS TO BE SEEN, NOT WHAT LIES UNDERNEATH. IT IS ONLY CONCERNED WITH STUDENT OUTCOMES. IT DOES NOT ACCOUNT FOR “INEQUITY & INEQUALITY,” “STRESS & VIOLENCE,” “SUPPORT FOR SCHOOLS,” AND “SUPPORT FOR YOUNG FAMILIES.”

Teachers and schools measured by EVAAS actually have to battle against all of the iceberg, not just the tip which is by far the smallest part of the iceberg.

The state pays more than three million dollars annually to SAS which was co-founded and is still run by Jim Goodnight who according to Forbes Magazine is one of the top donating executives to political campaigns. In 2016 he donated much to a PAC for Jeb Bush who while in Florida instituted the school performance grade system that North Carolina uses now – the same one that utilizes EVAAS reports to measure schools (https://www.bizjournals.com/triangle/blog/techflash/2015/10/forbes-sas-goodnight-among-tech-execs-for-top.html).

It also is worth looking at the fact that his wife, Ann Goodnight, is a co-founder and board member of BEST NC. When BEST NC had its annual legislative meeting it brought in the toxic Michelle Rhee and her campaign for value-added measurements to discuss policy. That “closed-door” meeting was held at SAS headquarters.

The recent principal pay schedule that has garnered well-deserved criticism was spearheaded by BEST NC with legislators behind the scenes over the summer utilizes EVAAS data.

Too much is being dictated by a private entity that is privately calculating data in a secret fashion to measure a public good and how much should be spent on that public good in a state that wants to privatize that public good.

There’s too much incestuous synergy there. And all of it is purposefully ignoring the part of the iceberg that is beneath the water line.

In using EVAAS, what the state of North Carolina is doing is sending schools on expeditions in remote icy waters without the use of radar and sonar to navigate themselves. It’s like the Titanic.

But instead of being surprised at the fact that the “unsinkable” actually succumbed to a lonely iceberg, the state has already made a hole in the hull for water to leak in, so even if the “ships” avoid hitting icebergs, they would already have a hard time reaching port. As the “unsinkable” ships begin to sink, the state says we must invest in other alternatives like charter school reform and vouchers, so the money starts going to other modes of “transportation.”

The problem is that the icebergs in our state are getting bigger and more are breaking off. As the income gap widens and as segregationist tendencies begin to take firmer root, systems like EVAAS will still serve as a façade of the actual truth which lies beneath the water.

Of course, SAS could release how it uses data and calculates its reports but that would require transparency.

But icebergs work best in cold, murky, choppy waters. And people in Raleigh like having big icebergs.

EVAAS, The Iceberg Effect, and Stranger Things – The Secretive Way NC Measures Teachers and Schools

This past month the venerable James Ford of the Public School Forum delivered the keynote address at the North Carolina English Teacher’s Association. It was more than exceptional as Ford highlighted that what hurts our schools most are external factors that are not being dealt with such as systemic poverty.

Part of his presentation included a version of what is called the “Iceberg Effect” for education. It looks like this:

iceberg

Ford talked about (and he is not alone in this belief) how what is above the water, namely student outcomes, is what drives educational policies in our state.

Notice that he means what is visible above the water line is what drives policy. That is what the public sees in the press. That is what lawmakers and leaders hark on when discussing what to do about public education. That is what is being used to measure the effectiveness of teachers and schools.

In 2013, the state of North Carolina started using a value-added measurement scale to help gauge teacher effectiveness and school performance. Developed by SAS which is headquartered in the Triangle area of Raleigh, Durham, and Chapel Hill, EVAAS collects student data and creates reports that are used to measure teacher and school effectiveness.

EVAAS stands for “Education Value-Added Assessment System.” For teachers, it is supposed to give an indication of how well students are supposed to do in a given year on the tests that are used on evaluations. (Do not let it be lost on anyone that “EVAAS” scores were just released at the end of most schools’ first quarter after half of the block classes have already completed more than half of the curriculum’s work).

EVAAS has been the subject of a lot of scrutiny. It deserves every bit of that scrutiny. Why? Because the algorithms that it uses to come up with its calculations and reports are like a tightly held secret.

Think of the Colonel’s special blend of spices on Kentucky Fried Chicken.

Think of the formula for Coca-Cola Classic.

Think of the recipe for Crabby Patties.

Think of whatever is happening Hawkins National Laboratory.

Hawkins_National_Lab

If you do not know the last reference, then watch Stranger Things on Netflix especially if you grew up in the 1980’s because they nail it.

Actually, think of the iceberg and what is seen and what is under the water line.

The North Carolina Department of Public Instruction describes EVAAS as:

EVAAS examines the impact of teachers, schools, and districts on the learning of their students in specific courses, grades, and subjects. Users can access colorful, easy-to-understand charts and graphs via the Web, as well as produce customized reports that predict student success, show the effects of schooling at particular schools, or reveal patterns in subgroup performance (http://www.ncpublicschools.org/effectiveness-model/evaas/).

There is even a nice little video that one can go to in order to “understand how EVAAS” works (https://ncdpi.sas.com/videos/EVAAS/WhatIsEVAAS.mp4).

EVAAS pic

The whole video is an attempt to validate the use of EVAAS by the state. Except it does not tell anyone how “EVAAS performs value-added analysis.” The only people who know how that works are inside of the Hawkins National Laboratory or as we know it, SAS headquarters.

This past March, Angela Scioli wrote a powerful piece for EDNC.org entitled “EVAAS: An incomplete and painful system for me.” In it she stated,

I did not change anything else about my teaching.  I did not know what to change.  No one met with me to intervene.  No one even spoke to me about the results.  It just sat there, like a black eye I couldn’t cover up, but no one wanted to talk about it.  

The next year, I received my EVAAS results, after using the same methods, and I was now deemed “highly effective.”  I was relieved and confused.  How could that be? (https://www.ednc.org/2017/03/21/evaas-incomplete-painful-system/).   

Justin Parmenter’s op-ed entitled “The cost of doing business in the education world” (August 9, 2017) was another powerful expose of a world in which EVAAS is being used to measure teachers and schools. He said,

In the years that followed, EVAAS was rolled out on a larger scale across the district and state, and similar data measuring teacher effectiveness was made available to more teachers. I was dismayed to see that, while some years I apparently had made a difference, there were other years when I did not make much of a difference at all. Some years I even made a negative difference (https://www.ednc.org/2017/08/09/cost-business-education-world/).

This criticism of EVASS is not limited to North Carolina. From the National Education Policy Center:

Education Policy Analysis Archives recently published an article by Audrey Amrein-Beardsley and Clarin Collins that effectively exposes the Houston Independent School District use of a value-added teacher evaluation system as a disaster. The Educational Value-Added Assessment System (EVAAS) is alleged by its creators, the European software giant SAS, to be the “the most robust and reliable” system of teacher evaluation ever invented. Amrein-Beardsley and Collins demonstrate to the contrary that EVAAS is a psychometric bad joke and a nightmare to teachers” (http://nepc.colorado.edu/blog/houston-you-have-problem).

And the ambiguity of how SAS uses data within the EVAAS program is not lost on many people. From a 2014 WUNC report called “Ranking Teachers: NC Bets Big On A Complicated Stats Model,”

EVAAS is based on that student growth, not the test score itself. And the software is complicated – and some say largely secret. Teachers, principals, even administrators at the state level don’t know everything that goes into the model.

“Now the statisticians, and I’m not a statistician – I’m not the smartest guy in the world – they would say that stuff should even out, and I think they are correct, I’m sure it does even out, when you look at statewide data,” says Jim Key, an assistant superintendent in Durham. “But within a particular classroom? You could have more than a normal share of students who are going through some challenges with their personal lives” (http://wunc.org/post/ranking-teachers-nc-bets-big-complicated-stats-model#stream/0).

That last quote from Mr. Key accurately sums up the relationship between the EVAAS program and the Iceberg Effect.

Simply put, EVAAS only measures what is the tip of the iceberg that is above the water and then it tells us how to view it. It completely disregards what is under the water level.

LET ME REPEAT: EVAAS ONLY MEASURES WHAT IS TO BE SEEN, NOT WHAT LIES UNDERNEATH. IT IS ONLY CONCERNED WITH STUDENT OUTCOMES. IT DOES NOT ACCOUNT FOR “INEQUITY & INEQUALITY,” “STRESS & VIOLENCE,” “SUPPORT FOR SCHOOLS,” AND “SUPPORT FOR YOUNG FAMILIES.”

Teachers and schools measured by EVAAS actually have to battle against all of the iceberg, not just the tip which is by far the smallest part of the iceberg.

The state pays more than three million dollars annually to SAS which was co-founded and is still run by Jim Goodnight who according to Forbes Magazine is one of the top donating executives to political campaigns. In 2016 he donated much to a PAC for Jeb Bush who while in Florida instituted the school performance grade system that North Carolina uses now – the same one that utilizes EVAAS reports to measure schools (https://www.bizjournals.com/triangle/blog/techflash/2015/10/forbes-sas-goodnight-among-tech-execs-for-top.html).

It also is worth looking at the fact that his wife, Ann Goodnight, is a co-founder and board member of BEST NC. When BEST NC had its annual legislative meeting it brought in the toxic Michelle Rhee and her campaign for value-added measurements to discuss policy. That “closed-door” meeting was held at SAS headquarters.

The recent principal pay schedule that has garnered well-deserved criticism was spearheaded by BEST NC with legislators behind the scenes over the summer utilizes EVAAS data.

Too much is being dictated by a private entity that is privately calculating data in a secret fashion to measure a public good and how much should be spent on that public good in a state that wants to privatize that public good.

There’s too much incestuous synergy there. And all of it is purposefully ignoring the part of the iceberg that is beneath the water line.

In using EVAAS, what the state of North Carolina is doing is sending schools on expeditions in remote icy waters without the use of radar and sonar to navigate themselves. It’s like the Titanic.

But instead of being surprised at the fact that the “unsinkable” actually succumbed to a lonely iceberg, the state has already made a hole in the hull for water to leak in, so even if the “ships” avoid hitting icebergs, they would already have a hard time reaching port. As the “unsinkable” ships begin to sink, the state says we must invest in other alternatives like charter school reform and vouchers, so the money starts going to other modes of “transportation.”

The problem is that the icebergs in our state are getting bigger and more are breaking off. As the income gap widens and as segregationist tendencies begin to take firmer root, systems like EVAAS will still serve as a façade of the actual truth which lies beneath the water.

Of course, SAS could release how it uses data and calculates its reports but that would require transparency.

But icebergs work best in cold, murky, choppy waters. And people in Raleigh like having big icebergs.