Measuring school performance

Theprospect of measuring school performance by testing students to nationalstandards has upset many teachers and educationalists. There seem to be threemain fears:

  1. Labelling achild as a failure is not going to improve their learning.
  2. Teachers willteach to the test, resulting in students receiving a narrower education.
  3. Schools will beunfairly compared with one another.

The firstconcern really has nothing to do with national testing and can be easilyaddressed in the classroom.   Especially at primary school level, the main rolefor testing is the formative assessment of student progress – an iterativeprocess of measuring how individual students are progressing and designingprogrammes that meet their individual needs and learning styles.   

Teaching tothe test is indeed a retrograde step if those tests are predictable andformulaic.   Modern testing systems such as asTTle (Assessment Tools forTeaching and Learning), however, consist of a databank of thousands of questionswith the option for teachers to select random questions stratified by degree ofdifficulty and by subject sub-component, such as algebra or comprehension.   Theprobability of two tests being the same is remote, meaning that ‘teaching tothe test’ as we currently understand it is virtually impossible.  

To aneconomist the third concern raised above is more interesting.   Comparisonsbetween schools need to be based on value-added, something that economists areused to measuring.   Value-added is a critical concept.   We don’t assess acompany’s performance by its gross value of sales.   We net out what it pays forits inputs so that we can assess how much value it is adding to those inputs.  Our standard of living depends on allocating resources to those industries thatcan use them most efficiently.   This applies to schools too.   As taxpayers wewant to see schools securing the highest possible academic achievement fromtheir students, thereby giving them more choice about career options andeventually contributing to raising our collective quality of life.   (I assumefor the moment that academic achievement is the sole criterion of a school’ssuccess.)

Comparingthe raw academic results of a decile 10 school with those of a decile 1 schoolis unlikely to tell you anything other than that the students at the former arelikely to have all the developmental and educational advantages that usuallyaccompany high parental income, educated parents, no overcrowding at home, andso on.  

Theaccompanying graph shows last year’s Level 1 NCEA pass rates for 32 colleges inthe Dominion Post’s circulation area.   The first interesting feature of thegraph is that a best fit trend through all the points is approximately linearand is upward sloping, providing a prima facie case that school decileis positively correlated with academic achievement.   The second interestingfeature is that there are some marked divergences from the trend.   For   exampleSchool 23 appears to provide much less value-added than would be expected purelyon the basis of its decile rating, whilst School 12, a decile 7 school,performs better than expected and indeed better than some decile 10 schools.

Hence afirst approximation to a school’s value-added is the distance between theactual result and the statistically fitted trend line.   But in this simplemodel all schools in a given decile are treated as having identical studentbodies, which is clearly not true.   Other potentially important factors shouldbe included in the model.   For example:

  • Insufficientgradation in the decile rating.   How similar are students in the 91stand 99th percentiles?
  • The variabilitywithin a school’s decile rating may be as important as the average.   Twoschools could have the same average decile, but one could be much morehomogeneous in terms of its mix of students than the other, perhaps providingan easier teaching environment.
  • Size of school -do larger schools allow gains from economies of scale in the delivery ofeducation, or does size detract from personal interaction between staff andstudent?
  • Single sex orco-educational.
  • Small samplelimitations – data across all secondary schools in New Zealand would yield morerobust inferences.
  • Variation overtime – a single year may be unrepresentative for any given school because ofparticular circumstances in that year.  

Accordinglya number of refinements would be required before an accurate measure of a  school’s value-added can be established.   This presents a challenge toeducation researchers and to the media who report exam results.   Trying to stoppublication of school test scores or making access to them difficult is not asolution.   Secrecy only instigates rumour and misinformation.   Far better tomake the information readily accessible and encourage researchers to competewith each other to see   who can estimate the most robust measures ofvalue-added and identify which schools really are performing best.   The recentpublication of data on hospital waiting times and other performance measures providesa parallel in the health sector.

And,returning to the earlier point about how school success should be defined, wemay see multi-dimensional analyses that incorporate not just a school’sacademic achievement, but how its students achieve on the sports field or incultural domains.  

Enjoyed this article?

You might like to subscribe to our newsletter and receive the latest news from Infometrics in your inbox. It’s free and we won’t ever spam you.