What Does Proficient Mean?
Politicians and government agencies use state tests to measure student ‘proficiency.’
But news stories frequently suggest these tests are too easy. Or contain basic mistakes. Or don’t cover all the topics your child is supposed to be learning. State tests are supposed to give parents guidance, but for many kids, the results aren’t precise enough to be useful.
The primary purpose of the state Math and English (ELA) exams given each spring is to judge if students have met New York’s proficiency threshold.
The number of questions your child answers correctly is transformed into a scale score which takes into account the difficulty of the questions. If she answered 47 out of 69 correctly on the New York State 4th grade ELA test in 2011, this would result in a scale score of 669.
The Education Department in Albany decides which scores correspond to the four levels shown below.
- Below Standard (1)
- Meets Basic Standard (2)
- Meets Proficiency Standard (3)
- Exceeds Proficiency Standard (4)
A scale score of 669 corresponded to Level 2 in 2011.
As the graphic shows there are more easy and medium difficulty questions, and fewer very challenging ones. This means the tests are less accurate for assessing the achievement level of high-performing students.
In this example, the entire range of “exceeds proficiency” was made up of just three questions. A scale score of 704 or 712, just below the cutoff for level 4, is statistically indistinguishable from a 722, which is above the cutoff.
Similarly, for the child scoring a 667, just one more correct answer would change her scale score to 671, putting her in the ‘proficient’ group. That one question difference doesn’t signal a true difference in her ability; it’s just an artifact of the test design.
New York City confuses matters further by calculating scores to two decimal places. So the 667 scale score becomes a 2.88; 671 is a 3.00. The decimals imply more accuracy than the tests truly provide. (Check your child’s account on ARIS).
There is much more detail about New York State tests on the Education department’s website, here. And if you’re stumped or just want to talk, drop us a note in the comment form to the right.
The Proficiency Illusion
As noted above, education experts determine proficiency based on their best judgment. Because the No Child Left Behind law of 2001 lets states select their own proficiency scores, political considerations encourage governments to set low hurdles. As early as 2007, one study pointed out that states were “aiming particularly low when it comes to their expectations for younger children, setting elementary students up to fail as they progress through their academic careers.”
Like many states, New York set low thresholds initially. As seen in the chart below (Click to enlarge), only 21% of children who were judged ‘proficient’ and graduated from New York City high schools, were able to do college level work.
Fewer children earned an Advanced Regents Diploma, which involves passing about eight Regents exams and taking a series of advanced classes. Even among more traditionally advantaged groups, only about half of the class of 2010 was prepared to succeed in college. These data are consistent with reports from The Educational Testing Service and the ACT, the two groups that administer standardized college achievement exams.
New York raised the proficiency score for 3rd through 8th graders in 2010 and lengthened the tests for 2012. Whether this is enough to ensure more students are on track for rigorous high schools and selective colleges remains to be seen.
Our informal feedback from college professors and district administrators indicates that the students inability to write effectively remains a significant concern.
High Achievers and Low Performers, Left Behind
NCLB’s worst impact has been on children who exceed proficiency benchmarks and on those who fall far short. Advanced students’ progress seems to have stalled, while struggling students may not be moving forward any faster than before 2001.
The Fordham Institute looked at higher performing students in a 2008 study. Researchers examined students’ scores on the National Assessment of Educational Progress (NAEP), as well as 900 teachers’ responses to a national survey.
Washington Post, March 4, 2007
Washington Post, March 4, 2007
Students scoring in the top 10% of the NAEP have made minimal gains in the NCLB era. Sixty three percent of teachers in the survey indicated that ‘struggling’ students received the most attention at their schools. Sixty-eight percent said these students were the prime interest of achievement tracking systems. Students who struggle were most likely to be the focus of one-on-one instruction according to 81% of teachers, while 5% of teachers said advanced students got this level of attention.
A 2007 study looked at low-scoring children in the Chicago public schools, concluding that “students who have no realistic chance of becoming proﬁcient in the near term appear to gain little from the introduction of these [NCLB-style] systems.”
According to that same study, test scores rose for children in the middle, presumably because the schools concluded that just a little more preparation could boost them over the ‘bubble’ to the next proficiency level. Other studies since 2007 have shown some positive results for some children; but state tests are at best blunt instruments.
NCLB rightly tried to call attention to the fate of children who struggle academically; for many years, if they did not have parents who advocated for them these kids were far too likely to be set aside or ignored.
Regardless of their child’s test scores parents must engage with their school and support their children to ensure their specific needs are being met. Relying exclusively or heavily on the results of the state exams to guide you in this process is insufficient.