Last year, there was widespread criticism of the plan by the Australian Curriculum, Assessment and Reporting Authority (ACARA) to take the NAPLAN persuasive writing test online. Two hand-picked Federal Government advisors said it would discriminate against disadvantaged students. It has also been widely criticised by teachers’ unions.
Professor Ken Wiltshire , who co-reviewed the national curriculum, said that online testing of writing “has serious implications for disadvantaged students” and that “a very large number of students will lose out”.
His co-reviewer, Kevin Donnelly, agreed:
Some schools and children could be disadvantaged because they just don’t have the family background or school resources in terms of being up to speed with computers and keyboards.
It seems they may have a case. A research study just published by the prestigious research arm of the US Department of Education, the National Centre for Education Statistics, found that testing writing by computer-based tests “appeared to widen the achievement gap” (p.vii).
The study compared the results from computer tests of written essays by 4th grade students in a 2012 pilot study with a pencil-and-paper test given to fourth graders in 2010. It found that high-performing students (those in the top 20%) scored much higher on the computer test than on paper. In contrast, low-performing students (representing the bottom 20%) as well as middle-performing students (representing the middle 60%) did not appear to benefit from using the computer. Low income, Black and Hispanic students are most strongly represented in the low performing group.
The differential effects of computer testing on the writing performance of high performing and other students is related to their prior exposure to writing on computers. Prior exposure includes having access to the internet at home, using the internet to look for information for writing, using the internet to write to friends and family and having received computer-based writing assignments at school.
The study reported that prior exposure to writing on computers has particular advantages in computer-based tests:
• text length tends to be longer (which is related to keyboarding skills);
• greater use of editing tools such as the spellcheck and backspace keys (which are related to editing words and sentences);
• preference for mode of writing (computer vs. paper).
It found that low performing students tended to use fewer words in responses to questions, not use spellcheck, not accept automated spelling corrections and use the backspace key less frequently.
The study found that one-quarter to one-third of students did not have access to a computer at home, never or hardly ever received computer-based writing assignments at school or never or hardly ever searched the Internet for information to include in their writing had much lower test scores than students with these characteristics.
Low access to and use of computers was found to be strongly linked to demographic characteristics.
The percentage of fourth-graders who did not have access to the internet at home was significantly higher for low income, Black, Hispanic, English Language learners (ELL students) and students with a disability. For example, 35% of students eligible for free school lunches had no access to a computer at home compared to only 10% of those not eligible for free lunches.
The percentage of fourth-graders who never or hardly ever received computer-based writing assignments at school was higher among low income, Hispanic and ELL students. The percentage of low income students who spent more than an hour a week writing on the computer at school was less than half that of other students.
The results of the study give weight to criticism of online testing of writing in Australia. Of course, the results cannot be simply applied to Australian circumstances. However, the study does serve to raise questions about how well ACARA has investigated the implications of online testing of writing for social equity.
There is high household penetration of computers and use of the internet in Australia. The ABS report on Household Use of Information Technology in 2012-13 shows that 96 per cent of households with children under 15 have access to the Internet. The OECD’s 2012 PISA study shows that 99% of 15 year-old students in Australia have a computer at home and 97% have access to the internet at home.
However, there is evidence that disadvantaged students in Australia have significantly less access to computers and the internet than high SES students. The Australian PISA report shows that 93% of students in the lowest SES quartile have access to the internet at home compared with virtually 100% in the highest SES quartile. Almost three-quarters of high SES students have access to multiple computers at home compared with less than one quarter of low students. The report describes this difference as “telling” because students in families with a single computer may have to compete for access and their time using it might be quite limited, whilst students in families with multiple computers are likely to have greater access and thus more experience with computers.
Several disadvantaged groups in Australia are consistently identified as having less access to the internet. These include low income families; families in regional, rural and remote areas, Indigenous communities and people with disabilities. ABS analysis commissioned by the Smith Family shows that 32% of children aged 5-14 living in the most disadvantaged areas do not have access to the internet at home, compared with 9% of those living in the most advantaged areas.
Other studies show that single parent households with dependent children under 15 lag comparable dual parent households by 15-16% for Internet and broadband access. Indigenous households in Central Australia are 76% less likely to have Internet access than non-indigenous metropolitan households. Exclusion is also concentrated in areas of socio-economic disadvantage such as urban fringe areas, public housing estates and for those with no fixed abode.
Students in disadvantaged households are less likely to use a computer and the internet for school work at home. The 2012 PISA report for Australia shows that only 62% of low SES students use a computer at home for homework compared with 89% of high SES students and only 53% of low SES students use the internet at home for schoolwork compared with 83% of high SES students. Similar gaps exist for Indigenous and remote area students.
Disadvantaged students are also less likely to have access to a computer and the internet at school than high SES students. The PISA study found that 8% of low SES students do not have access to a computer at school compared with 3% of high SES students and 10% of low SES student do not have access to the internet at school compared with 4% of high SES students. The gap is slightly larger for Indigenous students.
The study also found that students from disadvantaged socio-economic backgrounds use computers at school to a lesser extent than students from advantaged backgrounds. For example, 58% of low SES students browse the internet for school work compared with 72% of high SES students and only 31% do homework on a school computer compared to 45% of high SES students.
Given this divide in access and use of computers and the internet, the introduction of online testing of writing seems hasty and premature. Disadvantaged students generally have less experience using computers for school work than advantaged students, especially for writing tasks, and it may well have the effect of exacerbating the very large achievement gap in writing between rich and poor in Australia. The 2015 NAPLAN results show that low SES and remote area students in Year 9 are about four years in learning behind high SES students in writing while Indigenous students are about six years behind.
As Professor Wiltshire argued last year there needs to be further research done on the equity implications of online testing of writing:
The Naplan online experiment must be put on hold until proper longer-term research is conducted into its social and economic implications.
It seems that ACARA has been more concerned about cutting costs than the potential effects for low income, Indigenous, regional and remote, refugee and disability students. As Professor Wiltshire also said, “….it is as plain the on-line compulsion is to suit bureaucrats and assessors and to cut costs, with little consideration for the students and their families.”
Online testing of persuasive writing should be postponed until its equity effects can be assessed. At the very least, students should be given the option of taking the paper and pencil test.
Trevor Cobbold