In recent years, use of information and communications technology in classroom learning has increased massively. A new paper published in the Journal of Economic Literature provides a comprehensive review of studies of the impact of new technology on learning. It finds that some education technologies contribute to the development of cognitive and non-cognitive skills in some circumstances and some are not as successful. It recommends more detailed research on the mechanisms by which these technologies can improve learning and in what education situations.
The overall results of the review suggest that educators and policy makers should exercise considerable caution in selecting education technology products and not be seduced by the marketing campaigns of education technology companies. Relatively few education technology products have been thoroughly evaluated for learning and cost effectiveness. Educators need support in deciding which products offer the most potential for meeting for the learning needs of their school and classroom. Inadequate discrimination in the adoption of products can be highly wasteful financially and fail to improve learning.
As the authors state:
In the end, it should not be about the most popular product or even necessarily the technology itself, but about the best way to help students of all ages and levels learn. [p. 987]
There has been a huge investment in education technology associated with computers, mobile phones, and the internet. The proliferation of new technologies is something of a double-edged sword. While most agree that education technology can be helpful under some circumstances, there is no consensus among researchers and educators about what types of technology are best worth investing in, in which contexts, and for which populations. At the same time, some uses of technology could potentially be harmful, and many parents are growing increasingly concerned that screen time may distract from student learning and development or that technology could displace the role of teachers.
Furthermore, the use of education technology is occurring in a context of deep and persistent inequality in society. Despite expanding access to some technologies, the digital divide remains very real and very wide. For example, the study notes that while 98% of children in US households with incomes exceeding $100,000 per year have a computer at home, only 67% of children in households with incomes lower than $25,000 have them. Even when disadvantaged students can physically access technology, they often lack the guidance needed for productive utilization—a “digital-use divide”. Depending on design and implementation, education technologies could alleviate or aggravate existing inequalities. Equity considerations thus add another layer to the need for caution when implementing technology-based education programs.
The review attempts to shed some light on these issues by examining of studies of the introduction and impact of education technologies. It examines studies in four categories: access to technology; computer-assisted learning, that is, computer programs and other software applications designed to improve academic skills; technology-enabled behavioural interventions in education which include programs that aim to overcome psychological or sociological barriers to learning; and online learning.
The review examined 13 studies of access to technology. At the K–12 level, much of the empirical evidence suggests that giving a child a computer has limited impacts on learning outcomes, but generally improves computer proficiency and other cognitive outcomes. While access to technology likely improves the learning environment by expanding opportunities for learning, increased access alone does not seem to advance cognitive skill formation.
The review examined 31 studies of computer assisted learning (CAL) in developed countries. The programs broadly covered software systems that aim to help students practice particular skills. The types of CAL programs that have been empirically evaluated constitute only a small portion of the many CAL products on the market and used by schools. The results of the studies reviewed should not be generalized to other CAL products.
Of the 31 studies reviewed, 21 reported statistically significant positive effects on learning and the majority of these (16 of 21) were focused on improving maths outcomes. The paper noted that maths content may be particularly suited to personalized learning software that adapts to students’ learning levels. Such programs may be suited to mitigating teacher constraints in classrooms where there are a range of learning levels. This raises important questions for how teachers should be deploying software to meet the range of learning needs in the classroom. The effect of CAL programs for reading and learning were found have mixed effects, some with positive results and others with little improvement.
The consistent results of personalized software across different contexts suggests that this feature may be an effective mechanism that helps to overcome common educational challenges in classrooms with different learning levels. However, the paper concluded that it is important to consider the potential mechanisms of impact for each program and not overgeneralize to infer that the results from the studies imply that all CAL programs are promising. Much more research is needed to isolate the mechanisms for when and why certain CAL programs improve learning. Identifying CAL programs that successfully enhance teachers’ ability to deliver instruction could yield profound benefits.
Online courses build on a tradition of correspondence courses which have been common over many years for K-12 students and tertiary education. Online university courses are now prevalent in many countries. While these have exploded in popularity over the last decade, there is limited rigorous research on the effectiveness of online learning.
The review examined 17 studies of the impact of online courses. The large majority were conducted on post-secondary courses. They show that without some degree of face-to-face teaching, learning outcomes may suffer, leading to lower test scores for fully online courses relative to face-to-face courses. In contrast, blended learning courses that have both a face-to-face component and an online component have not been found to significantly underperform purely face-to-face courses.
Educators have increasingly attempted to leverage online learning in middle and high schools. The study identified only two experimental studies in schools, both of which tested the effect of offering an online version of algebra content. One of these was conducted in 15 high schools in the Chicago Public Schools system with the lowest rates of students passing algebra. However, students in the face-to-face course outperformed those in the online course. The evidence from the study indicated that teachers in the face-to-face course were better able to flexibly incorporate a range of topics, and thus were better able to accommodate and engage students.
The other study was of grade 8 students in schools that did not offer a full algebra course. It found that taking the online course significantly improved students’ algebra achievement at the end of grade 8 and increased the likelihood of participating in advanced course-taking sequences in high school. However, the majority of the schools participating in the study were small rural schools. It is not clear therefore whether the results can be generalised.
The review concluded that one of the main justifications for the potential usefulness of online courses is that they can improve access to programs for students that otherwise might have trouble accessing them.
Technology-based behavioural programs are designed to target non-cognitive skill development and help students make better choices and realise better long-term outcomes. The programs typically seek to overcome hurdles in the education process brought on by gaps in certain types of non-cognitive skills. The term “non-cognitive” skills generally refers to a wide range of social and emotional skills. However, the review was limited to studies that covered a much narrower set of skills such as time management, motivation, and resilience.
The review identified studies in four categories: seven studies on encouraging parental engagement in learning activities; 13 on improving information flows in secondary school; 19 on encouraging success in transitioning to and through college, and 15 on psycho-social interventions.
Six studies of technology-based programs aiming to increase the quantity and quality of time spent by parents practicing skills with their pre-schoolers and 1st-4th grade students found positive results for literacy. The review of 13 studies or technology-based programs aimed at improving school–parent information flows also found positive results.
The studies of programs for transitioning successfully to college covered a variety of measures. Some were successful in helping students and others were not. Five studies of so-called “nudge” campaigns that refer to interventions providing sustained efforts to guide, encourage, and/or remind participants about one or more aspects of college success showed they can be effective in improving decisions and task fulfilment about financial aid and college enrolment and completion.
Psycho-social intervention programs aim to overcome emotional barrier to learning and cultivate attitudes conducive to learning. They typically involve short reading and writing exercises that attempt to convince students that an individual’s intelligence develops over time and that new skills can be learned with practice rather than academic ability being immutable. The studies generally found positive effects across all stages of education. The review concluded that large-scale text message campaigns, are often incredibly cheap to carry out and hold great promise as a cost-effective solution to many challenges associated with behavioural barriers in education.
The review made several recommendations for future empirical studies of education technology.
First, replications of programs showing promise, such as CAL and technology-enabled behavioural interventions, are needed to better understand both the generalizability and the credibility of the current findings. The results warrant replication to see if the same effect is found in different contexts and with different populations of students, and over longer periods of time.
In the case of CAL, additional research is needed to understand to what extent the observed impacts are related to specific implementation models. Open questions also remain regarding the underlying mechanisms of effective CAL programs, specifically how the software interacts with teachers and current curriculum. It is important to discern the potential mechanisms of impact before judging the promise of a particular software. For example, to what extent does a software that claims personalization truly adapt to match lessons to a student’s learning level without requiring the teacher to recognize and redirect struggling students. The authors state that much more research will be needed to test mechanisms and impacts of newly emerging CAL products of varying subjects.
Second, given the promising evidence on technology-enabled behavioural interventions, research should prioritize understanding when nudges most effectively support non-cognitive skill formation by varying the timing and content of messages and testing how these models interact with other educational supports.
Third, more work to understand effects of online learning is also needed as rigorous evidence is sparse. Future research should test different types of online and blended courses relative to face-to-face. In particular, there is a need to disentangle why online learning on its own does worse than face-to-face teaching. It could be the result of the loss of interaction with teachers and peers, or it could be a lack of a structured environment for time management and study skills, or both.
Fourth, more research is needed on the key mechanisms of impact at different stages in the process of skill development and on measuring longer term outcomes. There is also a need to better understand how education technology impacts on children of younger ages as its role is likely to be different at these ages.
Finally, there needs to be more rigorous evaluations of popular education technology products. Education technology is rapidly changing. More research is needed to evaluate how underlying mechanisms rather than a specific product can advance learning.