Unpublished data provided to Save Our Schools by the OECD shows that nearly 4 in 5 Australian students did not fully try in the 2022 Programme for International Student Assessment (PISA). The figures show wide differences in student effort between countries, which call into question the validity of country rankings of PISA results. Even more importantly, the high proportion of students in Australia and many other countries not fully trying in PISA indicates a broader problem, namely increasing student disaffection with learning and school. This appears to be a crucial factor behind declining results in many OECD countries that is often ignored in the commentary on Australia’s PISA results.
The new figures show that 77% of Australian students didn’t fully try in PISA 2022. This was the equal 4th highest proportion in the OECD. It was also the equal 4th highest of the 81 countries and regions participating in the tests. Only Denmark (81%), Sweden (80%), Germany (80%), Switzerland (80%) and Belgium (78%) had a higher proportion of students who did not fully try (Chart 1). Norway, UK, Austria and Singapore had the same proportion as Australia. The average for the OECD was 71%. Türkiye had the lowest proportion (47%) amongst OECD countries.
Source: OECD, Forthcoming.
On an effort scale of 1-to-10, Australian students reported an effort of 7.2 on the PISA test. They reported that they would have put in an effort of “9” had the test counted towards their marks.
The proportion of Australian students not fully trying increased by 4 percentage points from PISA 2018. This was higher than the average increase in the OECD of 3 points.
Interestingly, a much larger proportion of Australian female students than male students did not fully try in PISA 2022. Some 81% of female students did not fully try compared to 71% of male students. More female than male students did not fully try in most OECD countries.
The OECD acknowledged that many students view PISA as a low-stakes assessment that has no consequence for their future education. As a result, there is no incentive to put in full effort. Many studies show that that students do not perform as well in regular low-stakes tests as when they participate in tests that count towards their grade.
Both PISA 2018 and PISA 2022 show large variations between countries in the proportion of students not fully trying. In PISA 2018, the range between the highest and lowest proportions in the OECD was 43 percentage points and in PISA 2022 it was 34 points. While the percentage increased in most OECD countries it also declined in some.
Variations between countries in the proportion of students not fully trying raises question about the validity of international comparisons of student achievement based on PISA. Less effort in tests leads to lower results and differences in rankings may be influenced by differences in student effort. As the OECD has acknowledged:
… differences in countries’ and economies’ mean scores in PISA, and comparisons between PISA 2022 results and results from prior assessments may reflect differences not only in what students know and can do but how motivated they were to do their best. Put differently, PISA does not measure students’ maximum potential but what students actually do in situations where their individual performance is monitored only as part of their group’s performance. [OECD, PISA 2022 Results (Volume I): The State of Learning and Equity in Education, Paris, 2023 p. 298]
The country rankings could be significantly different based on tests where students fully try or where the proportions of students fully trying is similar across countries. For example, a study by the US Bureau of Economic Research using PISA 2015 data and employing different estimates of the proportion on students not fully trying found significant differences in country rankings. It estimated that Portugal’s 2015 ranking in science in PISA 2015 would have improved by 15 places from 31st to 16th if students had fully tried. Sweden’s ranking would have improved 11 places from 33rd to 22nd and Australia’s ranking by four places from 16th to 12th. It concluded:
Using PISA scores and rankings as done currently paints a distorted picture of where countries stand in both absolute and relative terms. [p. 24]
The high proportion of students not fully trying in the PISA tests also raises the issue of whether changes in student effort over time contributed to the overall decline in Australia’s PISA results and those of several OECD countries since 2003. Unfortunately, there is no data on student effort prior to PISA 2018. However, the OECD’s PISA 2022 report suggests that lower student effort likely reflects lower engagement with learning and school. It said that this “might account for some of the negative trends observed in several countries, particularly in mathematics results” [p. 304]. A sense of not belonging at school can result in less motivation, effort and participation learning. It and lead to disaffection and active disengagement from learning.
There is evidence of growing disaffection with school in Australia and many other OECD countries. Australia has one of the highest incidences of student disaffection with school in the OECD. In 2022, 30% of students reported that they do not feel they belong in school (OECD, PISA 2022 Results (Volume II): Learning During – and From – Disruption, Table II.B1.1.4). This was the equal 5th highest out of 30 OECD countries for which a longer time series is available. Just over one-fifth (21%) of students reported that they feel like an outsider at school. This was also the equal 5th highest in the OECD.
Female students reported having a much lower sense of belonging in school than male students (OECD, PISA 2022 Results (Volume II): Learning During – and From – Disruption, Table II.B1.1.2). The sense of belonging amongst female students was the 3rd lowest in the OECD and the gap between Australian females and males was the 5th largest.
Similarly, the sense of belonging in school was significantly lower for disadvantaged than for advantaged students in Australia. The sense of belonging for disadvantaged students was the equal 6th lowest in the OECD and much lower than the OECD average.
Student disaffection with school has increased in many countries including Australia since 2003. The proportion of Australian students who feel they don’t belong at school increased fourfold from 12% to 30% between PISA 2003 and 2022, an increase of18 percentage points or by150%.This was the equal 3rd largest increase in the OECD behind the Netherlands and the UK and the same as in New Zealand [Chart 2]. Many other OECD countries also experienced significant declines in student engagement at school.
Source: OECD, PISA 2015 Results (Volume III): Students’ Well-Being. Tables III 7.4 & III 7.5; OECD, PISA 2022 Results (Volume II): Learning During – and From – Disruption. Table II.B1.1.4
Note: The change is from 2012 for Chile, Estonia, Slovenia and USA.
Other PISA data show that the percentage of Australian students who feel like an outsider at school increased from 8% in 2003 to 21% in 2022, an increase of 13 percentage points or by 163%. This was the equal 3rd largest increase in the OECD.
One of the OECD reports on PISA 2022 showed a strong relationship between changes in the proportion of students who feel they do not belong in school and student performance in mathematics. The relationship in Australia was the equal 3rd strongest of 37 OECD countries. The average mathematics score declined by 10 points on the PISA scale for every unit decrease in the index of sense of belonging. It was only exceeded by Colombia and the Slovak Republic (12 points) and Estonia and Hungary (11 points). The average change across the OECD was 7 points.
There also appears to be a close relationship between the sense of belonging and reading scores across most OECD countries. While there are exceptions, most OECD countries including Australia have experienced declining student engagement with learning and school and declines in PISA results since 2003 [Chart 3].
Source: OECD, PISA 2022 Results (Volume I): The State of Learning and Equity in Education; OECD, PISA 2015 Results (Volume III): Students’ Well-Being. Tables III 7.4 & III 7.5; OECD, PISA 2022 Results (Volume II): Learning During – and From – Disruption. Table II.B1.1.4
Many factors influence student achievement such as funding, teacher qualifications, teacher shortages, classroom, student absences and a range of out-of-school factors such as socio-economic status of families and neighbourhoods. Changes in these factors will affect the extent of changes in student achievement in different countries.
However, as the OECD has suggested, changes in student engagement at school appear to be a common factor in the declines in PISA results in most OECD countries. This raises major policy issues about the causes and proper policy responses. It is too much to expect that schools can resolve the widespread social alienation in Western countries, but they can help mitigate it with the right support. School can make a difference and is the best chance for many traumatised children.
Strategies to engage students, especially disadvantaged students, better in school and learning should be a key part of the new national schools reform agreement being negotiated between the Commonwealth and state/territory governments. Improving student well-being was a priority area identified by the report of the Expert Panel on the national schools agreement and it made a number of recommendations to improve student well-being.
The Expert Panel recommended, fully funding public schools as a priority given that they enol over 80% of low SES, Indigenous and remote area students. This is necessary to overcome the shortage of teachers and provide comprehensive student support measures including provision for more student voice and agency. Student support in school is fundamental to engaging students in learning. In the first instance, this is a funding issue because support for students involves more adults in a variety of roles in schools. A range of non-teaching professionals is needed to provide counselling, health, mental health, welfare, youth support services etc. Schools should be seen and resourced as community service hubs.
Improving student engagement with learning and consequent improvements in student outcomes in disadvantaged schools would have wide ranging personal, economic and societal benefits.