Navigation
ELLS Literature Review
 
 

Recent developments in Technology and language learning: A literature review and meta-analysis

Yong Zhao

College of Education

Michigan State University

 

Addres 115 EricksonCollege of EducationMichigan State UniversityEast Lansing, MI 48864
Phone 517-353-4325
Fax 517-432-4797
Email

zhaoyo@msu.edu

 

________________________

1This study was supported by a contract from the United States Department of Education as part of its E-language initiative. A version of this paper was submitted to the US Department of Education as a concept paper. The author wishes to thank Dr. Alan Ginsburg, Dr. Susan Sclafani, and Adriana de Kanter for their support and insights. Views expressed in this paper do not necessarily reflect those of the US Department of Education.

 


Abstract

 

This purpose of this study is to assess the potential of technology for improving language education. A review of the effectiveness of past and current practices in the application of information and communication technology(ICT) in language education and the availability as well as capacities of current ICTs was conducted.   The review found that existing literature on the effectiveness of technology uses in language education is very limited in four aspects: a) The number of systematic, well-designed empirical evaluative studies of the effects of technology uses in language learning is very small, b) the settings of instruction where the studies were conducted were limited to higher education and adult learners, c) the languages studied were limited to common foreign languages and English as a foreign or second language, and d) the experiments were often short-term and about one or two aspects of language learning (e.g., vocabulary or grammar). However the limited number of available studies show a pattern of positive effects. They found technology-supported language learning is at least as effective as human teachers, if not more so.


   


Recent developments in Technology Supported language learning: A review of the literature and meta-analysis

 

This review study intended to address three related issues in technology and language education. First, policy makers and the general public are interested in learning about the effectiveness of using technology in language education because they need that information to help decide future investment decisions regarding technology (President's Committee of Advisors on Science and Technology (Panel on Educational Technology), 1997) . Second, researchers and developers are interested in knowing what has been done and what we already know about using technology to enhance language learning. Such knowledge will hopefully guide their further explorations and development. Third, language educators want to know what works and what does not so that they can make informed decisions in selecting the appropriate technology to use in their teaching.

To answer these questions is not easy for a number of reasons. First, technology is an ill-defined concept that encompasses a wide range of tools, artifacts, and practices, from multimedia computers to the Internet, from video tapes to on-line chatrooms, from web pages to interactive audio conferencing.   These technologies vary a great deal in their capacity, interface, and accessibility. It is thus misleading to think the effects of video-tapes are the same as those of the on-line chatrooms just because they are all called “technology.” Second, the effects of any technology on learning outcomes lie in its uses. A technology may hold great educational potentials, but until it is used properly, it may not at all have any positive impact on learning. Thus the assessing the effectiveness of a technology is in reality assessing the effectiveness of its uses rather than the technology itself. Since most ICTs can be used in a variety of ways, some more effective than others, it is inappropriate to over-generalize the effectiveness (or lack thereof) of one way of using the technology to the technology itself. Third, to further complicate things, the effectiveness of an educational approach is highly mediated by many other variables —the learner, the task, the instructional setting, and of course the assessment tool. Thus even the same use of a particular technology in different instructional settings may result in different learning outcomes. 

Clearly it is unreasonable to expect any single study to tell us whether or to what degree technology is effective in improving language learning. However, a comprehensive review of many studies can get us closer toward an answer(e.g., (Cavanaugh, 2001; Chapelle, 1997; Lou, Abrami, & d'Apollonia, 2001; Salaberry, 2001) . With the help of a research method called Meta-analysis (Glass, 1977; Hedges & Olkin, 1985; Lyons, 1995a) , we can assess the effectiveness of technology uses in language education by  analyzing findings of numerous empirical studies. A carefully conducted review can also help us develop a map of past and current work in the field of technology and language education.  The map should reveal what we know, what we have done, what works, and what does not. This study attempted at such a review.

Focusing on the issues of effectiveness, this review attempted to achieve three goals: 1) assessing the overall effectiveness of uses of technology in language education through meta-analysis, 2) exploring patterns of recent efforts in using technology to improve language learning, and 3) identifying effective ways to use technology in language education.

 

Method

 

Selection of Studies

 

There is a long history of using technology to improve language learning (Salaberry, 2001) . The review presented in this paper is limited to research published in referred journals during the last five years, from 1997 to 2001. The decision to limit the review to this period of time was motivated by the concern for relevance. The primary purpose of this review is to seek evidence and ideas that will guide our future work, rather than paint a comprehensive historical picture of research in computer assisted language learning, which can be found in many existing publications (e.g., Chapelle, 2001; Levy, 1997; Salaberry, 2001) . Thus it is reasonable to focus on studies of technological applications that have the most relevance. Relevance is considered in two ways: a) technology and b) pedagogy. As we know, technology changes constantly and rapidly. The technological innovations that we are most interested in and will most likely to have an impact on language education in the future are multimedia computing, the Internet, especially the Web, and speech synthesis and recognition. These innovations were fairly recent development and efforts to apply them in language education occurred even later.  Thus focusing the research publications in the past five years should give us sufficient insight into the applications of these relatively new technologies.  There was also a major paradigm shift in the pedagogical and research focus of technology applications in language education recently (Chapelle, 1997, 2001; Pennington, 1996; Salaberry, 2001) —a shift away from traditional drill-and-skill Computer Aided Instruction (CAI) model toward multimedia, intelligent CAI, and integration models. Studies about these applications of these newer models appeared more recently as well.

Works included in this review were identified from five representative journals devoted to a) research on second/foreign language education and b) technology and language learning. 

 

Selecting Representative Journals

 

A four-step process was followed to identify the representative journals. First, a key word search using “Computer Assisted Language Learning” was performed on ERIC through FirstSearch. The search was limited by year, document type, and language. Only journal articles published in 1997 through 2001 in English were included. The search resulted in a total of 389 articles. Second, all articles that did not have the key word “Second Language” were excluded, which resulted in a 355 articles. Third, all articles appeared in non-peer reviewed, irregularly published, or practice-oriented journals or magazines were excluded, resulting in a total of 156 articles. The fourth step is to calculate the distribution of the articles and their sources. These articles were published in 22 different journals. The journals fall into three categories: a) technology and language learning journals that specifically address issues in applications of technology in language learning and education; b) language learning and education journals that address issues in language learning and education in general, of which technology application is a sub-issue; and c) educational technology journals that address issues in the application of technology in education in general, of which language learning and education is considered a sub-area of study. Figure 1 shows the distribution of articles by journal categories.

Figure 1

However in many cases a journal may have only one or two articles. Nine journals had more than three articles. (see figure 1).

Figure 2

Notes:

System=System: An International Journal of Educational Technology and Applied Linguistics. CALICO=CALICO Journal. Journal of the Computer Assisted Language Instruction Consortium. LLT=Language Learning and Technology; JECR=Journal of Educational Computing Research; FLA=Foreign Language Annals; TQ=TESOL Quarterly. MLJ=the Modern Language Journal; SLA=Studies in Second Language Acquisition; CMLR=Canadian Modern Language Review. Other=other journals.

As Figure 1 shows, most articles appear in journals devoted to educational technology, especially language educational technology. The three journals devoted to language learning and technology (System, CALICO, and LLT) published nearly 70% of all the articles. Further the articles appear mostly in two journals: CALICO and LLT (see Figure 2).

Figure 3

Based on this analysis, we first selected the CALICO Journal and Language Learning and Technology to represent studies published in journals devoted to technology and language learning. The Journal of Educational Computing Research was selected to represent journals devoted to educational technology. The Modern Language Journal and TESOL Quarterly were selected to represent journals in language learning and education. We selected these two over Foreign Language Annals because they are generally considered more research oriented than FLA. This selection represents 72% of all articles found to be related to computer assisted language instruction from the ERIC database during the period of 1997 through 2001. Considering that 13% of the articles are scattered in 12 other journals, this selection of journals should be considered as a reasonable representative sample of journals that may publish studies in technology and second language learning and education.

 

Selection of Studies

 

Having selected the journals, the researcher read through abstracts of all articles published in these five journals since 1997 to identify possible studies to include in this review. After identifying the possible articles. The researcher read through all identified articles to select the ones suitable for a meta-analysis. The following criteria were used for the selection:

1.     The article must report results of an empirical study or multiple studies on the effectiveness or effects of a technological application aimed to improving language learning. Informational articles describing the process of technological applications or products and opinion articles that present theories or review the literature were included as part of the review but not for the meta-analysis of effects.

2.     Technology was more broadly conceived than just computers because the convergence of media is an emerging trend in language education technology. Thus studies about the application of video, audio, computer-assisted instruction programs, the Web, computer-mediated communications, simulation, speech technologies, word processing, e-books, and grammar checkers were all included.

3.     The studies included for the final meta-analysis must have measures of improvement of language proficiency. Self-assessment of improvement or attitudinal surveys was not acceptable.

A total of 9 articles were found to meet the above criteria for inclusion in the meta-analysis although many more articles published in these journals deal with technological applications in language learning.  While the meta-analysis was conducted on only the 9 studies, the review draw upon discussions and findings from the other articles as well as a number of books published around the same period of time.

 

Results

 

The results section is organized into three parts. Part one presents an overview of the literature on technology applications in language education. Part two summarizes how technology has been used to support language learning and the effectiveness of these usages. Part three reports the results from the meta-analysis study intended to assess the overall effectiveness of technology-supported language learning. 


Overview of the Literature

 

The review of recent research on technology-supported language learning reveals a number of interesting points regarding existing research in this area. First, the total number of well-designed experimental studies on the effectiveness of technological applications in language learning is very limited. For instance, the four issues in Volume 16 of the CALICO Journal, which were published in 1998 and 1999, contain 10 feature articles. Only two of the 10 are met the selection criteria to be included in the meta-analysis. Of all the 51 feature articles published in Language Learning and Technology, one met the criteria. The majority of the articles are either description of cases of uses of technology in language education and processes of software development or theoretical discussions of principles of technological applications. Some of the limited empirical studies did not measure learning gains, relied solely on learner self-reports as measures of effectiveness, or were not well designed.

Second, the studies were limited to college level language learners. None of the 9 studies was conducted in K-12 settings. Only two of the 9 studies were not about college students—their p participants were military linguists, who were also adults.

Third, the target languages under consideration in the studies were also limited. French and Spanish were the most studied (10 out 16 or 63%). Other more commonly studied languages were English as a second language (ESL), German, and Arabic.

Fourth, most of the studies were about the application of a single application instead of systemic large-scale integration of technology. Only two of the 16 studies were about long-term technology integration in the language learning environment. Thus the treatment reported was also short-term, lasting from a few hours to a few weeks.

Lastly, the applications of technology to enhance language learning have been wide-ranging, both in terms of the types of technology used and the issues language education needs to address. The studies investigated the uses of most available technologies including video, audio, multimedia, communication, network, and speech technologies. These technologies have been applied to support the teaching of various aspects of the language learning including vocabulary, grammar, reading, writing, speaking, listening, and culture.

 

Uses and Effectiveness of Technologies in Language Education

 

The following paragraphs present detailed discussions of the specific applications that have been studied over the past few years. The discussion is categorized into four groups: access to materials, communication opportunities, and feedback, and learner motivation.

 

Providing Access to Linguistic and Cultural Materials

 

Access and exposure to engaging, authentic, and comprehensible yet demanding materials in the target language is essential for successful language learning.  However for many language learners, whether in classes or self-study settings, such access is often limited. Thus language educators have long looked at information and communication technologies (ICT) as possible solutions to this problem (see Egbert, Chao, & Hanson-Smith, 1999; Hanson-Smith, 1999; Salaberry, 2001) . The uses of ICT to provide better access and exposure to linguistic and cultural material fall into the following three areas.

 

Enhancing access efficiency through digital multimedia technologies.  Digital multimedia technologies were used to make access to learning materials more efficiently than print media or audio recorders because a) multimedia (visual, audio, and text) presentations can create stronger memory links than a single medium alone and b) digital technology allows instant and accurate playbacks, which helps the learner to access specific segments much more easily without spending time to locate them, which can be tedious and a waste of time (Hanson-Smith, 1999; Thorton & Dudley, 1996) . Shea (2000) compared the time students needed to complete their language learning tasks using captioned video versus interactive video disc(IVD) and found that the students using IVD completed the tasks significantly faster(p<.05). Labrie (2000) found that although students spent more time learning a set of French words on paper than those who studied on computer where they could hear a word pronounced and see a picture about the word, they did not learn more words. In another study, Nutta (1998) found that after spending the same amount of time (one hour per day for seven days) learning verb tenses in English in two conditions: attending a regular class and receiving instruction from the teacher vs. using a multimedia (audio, video, recording capabilities etc.) computer program, the ESL students using the computer program performed as well or significantly better (on 3 out of 6 measures, p<.10) than their counterparts attending the class.

 

Enhancing authenticity using video and the Internet. Video materials can bring natural and context-rich linguistic and cultural materials to the learner while the Internet enables the learner to access authentic news and literature in the target language, which can reflect current cultural changes more effectively than printed sources (Bacon & Finnemann, 1990; Hanson-Smith, 1999; Herron, Cole, Corrie, & Dubreil, 1999; Herron, Dubreil, Cole, & Corrie, 2000; Kitajima & Lyman-Hager, 1998; Lafford & Lafford, 1997; Lee, 1998; Weyers, 1999) . Weyers(1999) studied the effectiveness of authentic video on college Spanish students. He had one class of students watch a Mexican television show as part of a second semester Spanish class that met 60 minutes daily for a total of 8 weeks while the other class followed the regular curriculum without the video. He found that the video group’s performance on both listening comprehension and oral production to be significantly better than the regular group (p<0.01). The video group also outperformed their counterparts on other measures of communicative competence. Herron (2000) found that video also helped their first year college French learners develop significantly better understanding of the target culture. In another study, Green and Youngs (2000) substituted regular classroom instruction with Web activities one class period per week for beginning college French and German students. After a semester, they found that “the substitution of one class day for directed, pedagogically sound Web activities seems to have allowed the treatment groups to continue to progress toward their personal and professional goals and allowed them to learn language at a rate similar to that of their peers in the control groups. It also appears, in general, that the students had a positive experience using the Web. . .” (p. 108).

 

Enhancing comprehensibility through learner control and multimedia annotations.  Comprehensible input is necessary for language learning but useful learning materials must also contain enough unfamiliar materials (Krashen, 1985) . For language learners, especially beginning and intermediate ones, authentic materials are often beyond their language proficiency and may become incomprehensible without help. To enhance comprehensibility of spoken materials, full caption, keyword caption, or slow-down the speech rate have been found to be effective (Shea, 2000; Zhao, 1997) . Zhao(1997) found that the ESL students who were able to flexibly slow down or speed up the rate of speech had significantly better listening comprehension than those who did not (p<0.05). For reading materials, glossing or multimedia annotations have been effective means to enhance comprehension (Al-Seghayer, 2001; Chun & Plass, 1997; Johnson, 1999; Lyman-Hager, 2000) . Al-Seghayer (2001) compared ESL students’ vocabulary learning in different annotation conditions and found that “a video clip in combination with a text definition is more effective in teaching unknown vocabulary than a picture in combination with a text definition . . . The variety of modality cues can reinforce each other and are linked together in meaningful ways to provide an in-depth experience.”(p<0.001 ) (p. 225).

 

Providing Opportunities for Communication

           

Engaging in authentic communication in the target language is another essential condition for successful language learning yet such opportunities do not exist for most learners. ICT has again been used in many different ways to create opportunities for language learners to communicate in the target language (see Hanson-Smith, 1999; Kelm, 1998; Muyskens, 1998; Warschauer & Kern, 2000) . Efforts in this area can be summarized into two groups: a) interaction with the computer and b) interaction through the computer with remote audiences.

 

Interactions with the computer.  Communicative interactions can occur in either written or spoken language or a combination of both. At the simplest level, a computer program can generate utterances either orally or in writing that require the learner to respond by selecting an answer with a mouse click or providing simple writing responses (Hanson-Smith, 1999) . With the advancement of speech synthesis and recognition technologies (Ehsani & Knodt, 1998) , the learner can also carry on near natural conversations with a computer program around pre-selected and programmed topics (Jared Bernstein, Najmi, & Ehsani, 1999; Egan, 1999; Harless, Zier, & Duncan, 1999; LaRocca, Morgan, & Bellinger, 1999; Wachowicz & Scott, 1999) . The learner can also give either written or spoken command to a computer program in a simulation and game environment. The computer program would then perform the command (Holland, Kaplan, & Sabol, 1999; LaRocca et al., 1999) . Harless et al (1999) , for example, tested the effectiveness of a virtual conversation program in Arabic at the Defense Language Institute. The program enabled the students to interview virtual characters of native speakers orally with speech recognition technology. After interacting with these virtual characters for at least eight hours per day for four days, the participants’ reading and speaking skills increased significantly (p<0.05) while their listening skill increased “convincingly.” In another study, Holland and her colleagues (1999) found a speech-enabled interactive micro-world program that allowed the learners of Arabic to construct objects by speaking to the computer improved student motivation and oral output. (1999) .

 

Interactions with remote audiences through the computer.  Computer-mediated communication and teleconferencing technologies have been used to create authentic communication opportunities for language learners since the 1980s (Beauvois, 1997;  Pennington, 1996) . The uses of CMC technologies, such as electronic mail, bulletin-boards, and chatrooms have been found to have many benefits for language learners (Beauvois, 1997; Cahill & Catanzaro, 1997; Kelm, 1998; Salaberry, 2001; Warschauer, 1998) . CMC brings the much needed audience to the language learner (Johnson, 1999) . It also promotes more equal and better participation, leading to more output in the target language(Beauvois, 1997) (Gonzalez-Bueno, 1998) . It fosters negotiation and form-focused learning (Pellettieri, 2000) . CMC was also found to enhance the writing process and improve student writing (Schultz, 2000) . Although CMC communication is, in most cases, conducted in writing, it has been found to improve oral proficiency as well. For instance, Beauvois(1997) found that second year French learners who held their discussions online achieved better oral proficiency than those who discusse