Measuring outcomes in the Vocational Education and Training sector

Every January and February, tertiary education organisations are in the media offering well-meaning advice to school leavers about their options, whilst subtly (or not in some instances) trying to lead these young people to study with them.  This is often done by espousing the ‘student experience’ and the outcomes that each learner is likely to have once completed. This year was no different.

Steven Joyce also came out and highlighted many of the positive things that the National Government has done in the tertiary education space since 2008.  These include changing the governments incentives from ‘bums on seat’s’ (actually started with Labour, but National carried it on), removing low value qualifications, reducing funding on courses that have low completion rates and moving to a more outcomes based framework.

The overall direction of travel by Steven Joyce and co. is laudable.  The challenge however is moving from a simple outcomes model that is about learners completing qualifications to a model that focuses on learners getting the skills that will benefit them (and employers, let not forget them) as well as contribute to social and economic wellbeing.

Many of these above issues are highlighted in the recently released NZ Productivity Commission issues paper as part of their long awaited inquiry into new models of tertiary education.  This paper, among other things, outlines and describes the main participants in tertiary education (students, providers, government and employers) as well as desired outcomes and the linkages between tertiary education and these outcomes.

The linkage between tertiary education and desired outcomes and how this aligns with Government funding incentives is something the Tertiary Education Commission and various other agencies have grappled with for quite a few years now.  For the past year (at least) it has been talking about its new ‘Investment Approach’.  Truth be told however, beyond the rhetoric and high level directional statements, the tertiary education sector still doesn’t really know what it means or could look like, let alone how it could be measured and incentivised.  In this regard, the NZ Productivity Commissions inquiry will be very useful.

Unsurprisingly, historically the behaviour of many organisations in the tertiary education sector has been influenced through fear of missing out on Government funding.  Government funding is also unfortunately structured in a way that often drives tertiary education organisations to compete against each other rather than collaborate in the best interest of learners and employers.  Yes, this is a gross over-generalisation, and there has been a lot of positive collaborative behaviour in this area, particularly in the non-university sector.

I was pleasantly surprised then to see that the peak bodies for the Institutes of Technology and Polytechnics (ITPs) and the industry training (ITO) sector put out a discussion document earlier this month outlining, at a broad level, how the system would ideally operate to ensure it contributes to better outcomes for learners, employers, community and the wider economy.  These are two parts of the tertiary education sector that only a few years ago were actively competing to get more government funding at the expense of the other and shift the Government’s performance focus away from them.

Framed as a discussion document for the vocational education and training (VET) in NZ, the document outlines potential aims, uses, and outcomes for a VET outcomes framework.  It does this by outlining ideal outcomes from the system, which is drawn on by international research and best practice.  This is best summarised in the following diagram:

Proposed outcome framework
wp-Proposed-outcome-framework.png

The discussion document then goes on to outline some potential broad outcome measures, such as workplace productivity.  This is always a tricky area, especially when there is little regular detailed robust data collected and the linkage between tertiary education and productivity is a lot less clear-cut that one would initially think.  For example, as the NZ Productivity Commission outlines, “In theory the education system should contribute to productivity improvements … but the evidence of actual productivity gains from education is weak”.   

Another outcome linkage that the VET discussion document makes is that of tertiary education and employment, with the Household Labour Force Survey (HLFS) proposed as a good measure of quarterly labour market data including the employment rate, the labour force participation rate and the unemployment rate.  While the HLFS is one of New Zealand’s key official statistics datasets, the linkage between the outcomes in the VET sector and measures including unemployment is not direct and is tenuous at best.

For example, economic conditions such as employer confidence, the exchange rate, and migration flows have a much more direct impact on both unemployment and employment.

Whilst the two papers have different focuses, the areas around outcomes appear to diverge.  This is unfortunate for the VET working group, whose paper came out first.  As the NZ Productivity Commission inquiry is likely to hold more sway with Government, I would recommend that the VET working group examine how the outcomes and discussion around them could be further aligned.  In addition to this, I would encourage the VET working group to not only identify existing indicators and measures but also to evaluate their strengths and weaknesses as well as identify ideal measures.  I feel this would signal to the government where the sector wants to go and potentially result in some new measures being developed.

The above said, I think the approach taken by the VET sector is really encouraging.  Getting past self-interest is tough, but the collaborative approach that has been taken combined with the potential coming out of the NZ Productivity Commission inquiry does provide some real substance for the TEC to start to run with in their ‘Investment Approach’.

You can view the VET working group discussion document here:  http://www.itf.org.nz/files/9314/5501/3104/VET_outcomes_discussion_document.pdf  

The NZ Productivity Commission issues paper on New Models of Tertiary Education can be viewed here: http://www.productivity.govt.nz/sites/default/files/tertiary-education-issues-paper.pdf  

Related Articles