An evaluation report on the fast-track teacher training program, Teach for Australia (TFA), raises serious questions about the effectiveness of the program. It shows that TFA teachers are not being placed in genuinely disadvantaged schools and a high proportion leave teaching within three years of completing the program. It calls for changes to increase retention such as longer placement lengths, or incentives for TFA teachers to stay in disadvantaged classrooms. There are also serious questions about the cost effectiveness of TFA and its impact on student outcomes.
TFA aims to fast-track high achieving university graduates into teaching. Selected applicants are supported to undertake an intensive six-week teacher training course and agree to teach in a disadvantaged secondary school for two years.
The objectives of TFA include attracting new high-quality entrants to the teaching profession to work in disadvantaged schools, retain a proportion of these new entrants beyond the initial two-year commitment, and achieve measurable increases in students’ academic achievement in disadvantaged schools. Two evaluation reports show that TFA is failing in its mission.
The program is largely funded by the Commonwealth Government. According to data provided to Senate Estimates by the Commonwealth Department of Education, the Commonwealth Government has provided $57 million for the program from 2008–09 to 2017–18 and has announced further funding of $20.5 million to 2020–21 [Answer to Question on Notice No. SQ17-000482, Additional Estimates 2016-2017]. Other data provided to Senate Estimates by the Department shows that 276 graduates had completed their 2-year placement by the end of 2016 [Answer to Question on Notice No. SQ17-000481, Additional Estimates 2016-2017].
TFA is not targeting the most disadvantaged schools
Information provided in the evaluation report indicates that TFA is failing in its mission to support disadvantaged students. Its latest annual report states that TFA is “dedicated to breaking the cycle of disadvantage”, while its Director’s report for 2016 says “We partner with schools serving low socioeconomic communities, exclusively”. The key objective of Commonwealth Government funding of TFA is to provide more high quality teachers in disadvantaged schools.
However, the evaluation report found that most TFA teachers are placed in schools just below the median of the Index of Community Socio-Educational Advantage (ICSEA), rather than in Australia’s most disadvantaged schools. It states:
Placements are mostly below, but not significantly below, the national median based on ICSEA. Placement schools tend to be clustered immediately below the median, rather than in Australia’s most disadvantaged schools. [p. 14]
Further, 13% of placements were in schools that are above the national median. Unfortunately, the report does not provide any data on the distribution of TFA placements by school ICSEA values or by location.
Thus, TFA appears to be ignoring the more disadvantaged schools in Australia. There is a huge difference in the proportion of low socio-educational advantaged students in schools just below the median ICSEA value and those at lower values. For example, data from the My School website shows that students from the lowest SEA quartile accounted for an average of 33% of enrolments in secondary schools with an ICSEA value in the range of 975 to 999 in 2016 compared to 59% in schools in the 900 to 925 range and 66% in schools in the range of 800 to 899.
Other data from My Schools shows that the average Year 9 NAPLAN results (across all test domains) for schools in the ICSEA range of 800 to 899 in 2016 were about two years of learning below those in the ICSEA range of 975-999. Results for schools in the ICSEA range of 900-925 were about one year behind those in the 975-999 range.
Why a program funded almost entirely by government to support disadvantaged schools should only target schools just below the median ICSEA value rather than more highly disadvantaged schools is a mystery. Greater accountability is needed from TFA and the Commonwealth Government about decisions on its school placements.
TFA has a high attrition rate
It also appears that TFA is compounding the serious problem of teacher retention in disadvantaged schools rather than reducing it. The report shows that the retention rate of TFA teachers is low and many of those that remain in teaching shift to more advantaged schools after their 2-year placement in a school. Less than 50% of TFA teachers are still teaching three years after completing their 2-year placement (that is, after five years) and only 30% remain in schools below the national median ICSEA value [p. 16]. It also estimated that the average TFA Associate spends 4.7 years teaching including their two-year placement, of which 3.2 years are in a school below the ICSEA national median.
The report found that school principals are very concerned about low retention of TFA teachers. Principals who do not intend to hire TFA teachers again cited concerns about retention and want a greater return on the investment they make in training and development. Principals who are enthusiastic about the TFA program also want greater retention. A range of stakeholders reported that other teachers are frustrated by low TFA retention, and see this as demonstrating a lack of genuine commitment.
The attrition rate of over 50% far exceeds most estimates of attrition by new teachers entering the profession through traditional teacher training. According to a Productivity Commission report, only about 10% of teachers in NSW leave the profession after five years [p. 63]. A report on attrition of Queensland graduate teachers published by the Queensland College of Teachers found an attrition rate of 15% within five years. Others have suggested that as many as 25 per cent of beginning teachers leave the profession within their first five years of employment, but there is no hard evidence of such an attrition rate. Even if true, it is only half the TFA attrition rate.
The high TFA attrition rate raises fundamental questions about the worth of the program. High attrition undermines the objective of improving school outcomes for disadvantaged students. Research studies in the United States and the United Kingdom have found that high teacher turnover negatively impacts on student achievement and the negative effects are strongest in disadvantaged schools. The primary reason for the negative impact is that high teacher turnover disrupts teacher-student relationships and staff cohesion.
Moreover, the high attrition rate is likely to impose significant financial and human resource costs on disadvantaged schools that employ TFA teachers. Recruiting, hiring, and training of new teachers requires significant financial and time costs that drain resources that might otherwise be spent on program improvement, teacher development and learning resources. For example, remaining teachers in a school may have less access to professional development resources because of the available resources are devoted to new teachers replacing TFA teachers.
All this suggests that the high attrition rate of TFA is undermining its goal of improving student outcomes in disadvantaged schools while also serving as a drain on the financial and human resources of those schools.
No objective assessment of the impact of TFA on student outcomes
The report fails to provide any data or analysis of the impact of TFA on student academic and social outcomes. Its assessment is based on interviews and opinion surveys of TFA teachers and school principals.
Its survey of TFA teachers found that they report variable performance among other early career teachers and believe that they are consistently among the best. They attribute this to a combination of their consistent enthusiasm and commitment and the specific TFA program supports they receive. However, this is a highly biased survey. It is hardly objective evidence of the impact of TFA teachers. Did the consultant really expect that TFA teachers would say that they do not perform as well as other early career teachers? The evaluation did not survey other early career or other teachers on their opinions about the relative performance of TFA and other teachers.
In interviews, principals said that TFA and other early career teachers were similar. However, the survey of principals found that TFA teachers outperformed other early career teachers on professional standards. However, the results are highly dubious because they are given an unwarranted degree of precision by the report.
The report states that it did not seek to use student outcomes as a measure of impact because of the lack of data at an individual class/teacher level. This is a strange excuse because part of the job of teachers is to record progress by students. The report says that it was unrealistic to collect data on outcomes because there would be reservations about establishing causation and lag times. However, the same problems arise with opinion interviews and surveys. They are not a reliable alternative to data on student outcomes.
There is no substantive evidence that TFA improves student outcomes any more than traditionally trained early career teachers. The failure to collect data on student academic and social outcomes is a consistent feature of evaluation reports on TFA. No provision was made at the inception of the program to collect data on student results even though a key objective of the program is to achieve measurable benefits for students in socially and educationally disadvantaged schools. The lack of such data inhibits an objective evaluation of the impact of TFA.
TFA is not cost effective
The report also failed to assess the cost of effectiveness of TFA. It said that it was unable to access detailed TFA cost information. This is very surprising as assessing cost effectiveness is a key aspect of any program evaluation.
Earlier evaluations by the Australian Council for Educational Research (ACER) had no such difficulty. The 2012 Phase 2 evaluation report found that it cost approximately $216,500 per TFA Associate, based on figures for Cohorts 1 and 2 of the program, compared to comparative Victorian post-graduate pathway of around $140,200 per fully registered teacher. It concluded that the “TFA Pathway is a relatively costly teacher education option for government” [p. xiii].
The Phase 3 evaluation report by ACER found that the cost of TFA training is over four times that of university-based training. It compared the cost of traditional and TFA training for 50 teachers. It estimated that the cost to government of training 50 teachers would be approximately $713,400 and the cost to students would be a maximum of $423,600. The total cost of training would be about $1.14 million over the training period. In contrast, the cost of TFA training for 50 teachers was estimated at $4.9 million. It concluded:
In terms of teacher education the TFA Pathway involves relatively high financial outlays by government. These relatively high costs are linked to the key elements of the Pathway, as well as the costs of establishing the Pathway and the relatively small number of Associates involved.
….it remains the case that the cost of recruitment, a quality teacher education component and Associate support is high relative to other pathways into teaching. [p. xvii]
There were some incompatibilities in the cost estimates, but they do not account for the cost disparity. The cost of the university pathway includes the base cost of the teaching course but does not include additional costs to schools of the practicum component of the course, any school-based coordination and the role of supervising teachers. These items are included in the TFA cost. However, they account for only about $1.2 million of the $3.8 million cost differential.
It is also apparent that a large proportion of TFA entrants would have undertaken teacher training without TFA. The ACER evaluation found that about one quarter of TFA teachers had already decided to enter teaching and nearly half may have entered teaching via a university graduate course had they not been successful in their TFA application [p. xiv]. Given the high cost of TFA, this involves a significant waste of taxpayer funds on TFA. This issue was not canvassed by the latest evaluation report.
There is also evidence that TFA is not using its taxpayer funding efficiently. The latest audited financial report on TFA shows expenditure on TFA staff and contractors of just under $5 million in 2016 while expenditure on teacher education was $2.6 million. TFA staff costs accounted for 55% of total expenditure in 2016 compared to 29% for teacher education. Staffing costs appear excessively high. The Phase 3 evaluation report by ACER noted that there are a high number of TFA staff working in recruitment of applicants [p. 100].
Clearly, there are ongoing questions about the cost effectiveness of TFA. A thoroughgoing public review of its cost effectiveness should be undertaken.
Questions about the tender process for the evaluation contract
The evaluation was conducted by the management consultancy firm Dandalo Partners. The contract for the evaluation was subject to a limited tender. This means that it was the result of a direct approach by the Commonwealth Department of Education to only one or more potential suppliers. In contrast, the 3-phase evaluation conducted by ACER was an open tender contract. The Commonwealth Department of Finance website states that limited tender contracts are only resorted to in the following circumstances:
• cases of extreme urgency beyond the control of the agency;
• where an unsolicited and highly-advantageous opportunity arises, representing an unusual value for money opportunity;
• where the goods or services can be provided by only one supplier, like a work of art;
• for additional deliveries of property or services for reasons of compatibility with existing equipment or services.
None of these criteria appear to justify the Government’s resort to a limited tender for the evaluation of TFA. It raises the question of government probity and whether the Government targeted a favoured contractor, given that the previous evaluation was subject to an open tender. The Minister for Education should explain why the Government resorted to a limited tender. He should also explain why the Government decided to extend funding for TFA in December last year before the evaluation was completed in May this year.
Conclusions
The new evaluation of TFA is very poor quality. It fails to assess whether TFA is achieving the key objectives of increasing student outcomes at school and whether it is cost effective. There is no financial analysis of the program. It doesn’t even provide basic data such as the number in each TFA cohort completing the program and the cost of training them.
However, the report does indicate that TFA is primarily targeting marginally disadvantaged schools rather than highly disadvantaged schools and that TFA has a high attrition rate. The attrition rate is much higher than that reported by other studies for traditionally trained teachers. This high attrition rate undermines the objective of improving results for disadvantaged students and adds to the resource drain on disadvantaged schools. There is also robust evidence from previous evaluations that TFA is not cost effective. Moreover, many joining TFA would have gone into teaching anyway. There is no substantive evidence that TFA is having a greater effect on student outcomes in disadvantaged schools than traditionally trained early career teachers.
In conclusion, the available evidence suggests that TFA is failing in its mission. Disadvantaged schools, disadvantaged students and the taxpayer would be better served by using TFA funding in more effective ways. There are alternative options for encouraging entry of high-achieving students into teaching. For example, the Commonwealth and State/Territory Governments could offer bonded scholarships for university courses conditional on teaching in disadvantaged schools for a period.This approach is used by the Commonwealth to address doctor shortages in rural areas through the Medical Rural Bonded Scholarship scheme.
Trevor Cobbold