Category: League Tables and Testing

The OECD’s Programme for International Student Assessment (PISA) has extraordinary status and influence. It is seen as the gold standard for assessing the performance of education systems, but it is a castle built on sand. New data published by the Australian Council for Educational Research (ACER) call into question the accuracy and reliability of PISA and its league tables of country results.

The new figures show that nearly three-quarters of Australian students didn’t fully try on the PISA 2018 tests. The ACER research found that “…the majority of Australian students (73%) indicated that they would have invested more effort if the PISA test counted towards their marks”.

Continue reading “Question Mark Over the Accuracy and Reliability of PISA Tests”

Test-based accountability has been a key education policy in most OECD countries, including Australia, for many years. It was believed that publication of school test results would put pressure on schools and teachers to increase student achievement. A new OECD study shows this policy is an abject failure. It found no evidence that test-based accountability has affected education outcomes in higher income countries.

Our results suggest that across most OECD countries test-based accountability does not relate to academic achievement, nor has a substantial impact on educational inequality for the subject of mathematics. With some small variations we achieved similar results for the subjects of reading and science. [p. 25]

The findings have important implications for policy makers in higher income countries:

Continue reading “OECD Says Publication of School Results Has Failed to Improve School Performance”

The international PISA tests have become “false idols of educational excellence for the world to worship”. They have extraordinary status and influence. Education systems and even nations are judged by test scores and rankings that are assumed to be accurate measures of achievement. A primary assumption is that all students always try their best. A growing literature shows this to be false.

The OECD report on PISA 2018 found that over two-thirds of students in the OECD did not fully try on the tests. Eighty per cent of students in Germany did not fully try as did 79% of students in Denmark and Canada and 78% in Switzerland. Some 73% of Australian and New Zealand students did not fully try. In contrast, 46% of Korean students and 60% of Japanese students did not fully try. This variation calls into question the validity of league tables of countries based on PISA results.

Continue reading “Beware False Idols of Education Excellence”

One factor not considered in the commotion over the continuing decline in Australia’s PISA results is whether students try their best on the tests. The OECD’s own report on PISA 2018 shows that about three in four Australian students and two-thirds of students in OECD countries did not try their hardest on the tests. There are also wide differences between countries. It has potentially explosive implications for the validity of international comparisons of student achievement based on PISA.

The PISA data also shows increasing student dissatisfaction with school which likely contributes to lack of effort on tests and is a factor, among others, behind Australia’s declining results. There is also a perplexing contradiction between Australia’s declining PISA results and its improving Year 12 results. Lack of effort in PISA may partly explain this because performance on PISA has no consequences for students as they don’t even get their individual results. In contrast, Year 12 outcomes affect the life chances of students and even students dissatisfied with school have greater incentive to try harder. The fact that Australia’s Year 12 results have improved significantly since the early 2000s raises further questions about the reliability of the PISA results.

Continue reading “OECD Says 3 in 4 Australian Students Do Not Try on PISA Tests”

A US high school student takes issue with standardised tests.

Continue reading “A Poem on the Ravages of Standardised Tests”

How often have we all sat through those frustrating meetings where someone from head office or a university articulates with such commitment the first lie – if you can’t measure it then it’s not worth doing.  This quantification of education based on an economically rational approach started in the sixties.  This was the dawn of outcomes-based learning. 

Continue reading “The Dishonourable Lie”

This is a summary of a new Education Research Brief. It can be downloaded below

A much-ignored aspect of school results in Australia over the past decade or more is the sharp contrast between declining or stagnating scores on international and national tests for Years 9 and 10 and solid improvements in Year 12 results. How is it that trends in school outcomes only two or three Year levels apart are so different? Continue reading “Have Kids Stopped Trying on PISA and NAPLAN?”

National literacy and numeracy tests will now have ‘high stakes’ attached to them as a result of the decision of Australian education ministers, at the initiative of the Rudd Government, to publish the results of individual schools.

It means that league tables are now inevitable in Australia. This will put schools under enormous pressure to maintain reputations and enrolments. The future of some schools will also be threatened because the Prime Minister has stated that sanctions will be applied to schools that don’t improve their performance. Continue reading “League Tables Create Incentives for Schools to Rig Their Results”

Last year, the Director of Education at the OECD, Andreas Schleicher, admitted that the switch from pen-and-paper to computer tests for PISA 2015 assessments may have contributed to significant falls in results amongst higher performing countries. A new research paper published by the Centre of Education Economics in the UK provides more evidence for this. Continue reading “Caution Needed in Interpreting PISA 2015 Results”

The introduction of a National Year 1 Literacy and Numeracy Check has been heavily criticised by the Australian Literacy Educators’ Association (ALEA) in a position statement. It says that there is an unreasonable over-emphasis on phonics in the new assessment tool. Continue reading “Does Australia need an assessment tool to measure literacy and numeracy achievement in Year 1 classrooms?”