Misplaced Pages

Computer-assisted language learning

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
(Redirected from Foreign language reading aid) Learning technique

Computer-assisted language learning (CALL), known as computer-aided instruction (CAI) in British English and computer-aided language instruction (CALI) in American English, Levy (1997: p. 1) briefly defines it as "the exploration and study of computer applications in language teaching and learning." CALL embraces a wide range of information and communications technology "applications and approaches to teaching and learning foreign languages, ranging from the traditional drill-and-practice programs that characterized CALL in the 1960s and 1970s to more recent manifestations of CALL, such as those utilized virtual learning environment and Web-based distance learning. It also extends to the use of corpora and concordancers, interactive whiteboards, computer-mediated communication (CMC), language learning in virtual worlds, and mobile-assisted language learning (MALL).

The term CALI (computer-assisted language instruction) was used before CALL, originating as a subset of the broader term CAI (computer-assisted instruction). CALI fell out of favor among language teachers, however, because it seemed to emphasize a teacher-centered instructional approach. Language teachers increasingly favored a student-centered approach focused on learning rather than instruction. CALL began to replace CALI in the early 1980s (Davies & Higgins, 1982: p. 3). and it is now incorporated into the names of the growing number of professional associations worldwide.

An alternative term, technology-enhanced language learning (TELL), also emerged around the early 1990s: e.g. the TELL Consortium project, University of Hull.

The current philosophy of CALL emphasizes student-centered materials that empower learners to work independently. These materials can be structured or unstructured but typically incorporate two key features: interactive and individualized learning. CALL employs tools that assist teachers in facilitating language learning, whether reinforcing classroom lessons or providing additional support to learners. The design of CALL materials typically integrates principles from language pedagogy and methodology, drawing from various learning theories such as behaviourism, cognitive theory, constructivism, and second-language acquisition theories like Stephen Krashen's. monitor hypothesis.

A combination of face-to-face teaching and CALL is usually referred to as blended learning. Blended learning is designed to increase learning potential and is more commonly found than pure CALL (Pegrum 2009: p. 27).

See Davies et al. (2011: Section 1.1, What is CALL?). See also Levy & Hubbard (2005), who raise the question Why call CALL "CALL"?

History

CALL dates back to the 1960s, when it was first introduced on university mainframe computers. The PLATO project, initiated at the University of Illinois in 1960, is an important landmark in the early development of CALL (Marty 1981). The advent of the microcomputer in the late 1970s brought computing within the range of a wider audience, resulting in a boom in the development of CALL programs and a flurry of publications of books on CALL in the early 1980s.

Dozens of CALL programs are currently available on the internet, at prices ranging from free to expensive, and other programs are available only through university language courses.

There have been several attempts to document the history of CALL. Sanders (1995) covers the period from the mid-1960s to the mid-1990s, focusing on CALL in North America. Delcloque (2000) documents the history of CALL worldwide, from its beginnings in the 1960s to the dawning of the new millennium. Davies (2005) takes a look back at CALL's past and attempts to predict where it is going. Hubbard (2009) offers a compilation of 74 key articles and book excerpts, originally published in the years 1988–2007, that give a comprehensive overview of the wide range of leading ideas and research results that have exerted an influence on the development of CALL or that show promise in doing so in the future. A published review of Hubbard's collection can be found in Language Learning & Technology 14, 3 (2010).

Butler-Pascoe (2011) looks at the history of CALL from a different point of view, namely the evolution of CALL in the dual fields of educational technology and second/foreign language acquisition and the paradigm shifts experienced along the way.

See also Davies et al. (2011: Section 2, History of CALL).

Typology and phases

During the 1980s and 1990s, several attempts were made to establish a CALL typology. A wide range of different types of CALL programs was identified by Davies & Higgins (1985), Jones & Fortescue (1987), Hardisty & Windeatt (1989) and Levy (1997: pp. 118ff.). These included gap-filling and Cloze programs, multiple-choice programs, free-format (text-entry) programs, adventures and simulations, action mazes, sentence-reordering programs, exploratory programs—and "total Cloze", a type of program in which the learner has to reconstruct a whole text. Most of these early programs still exist in modernised versions.

Since the 1990s, it has become increasingly difficult to categorise CALL as it now extends to the use of blogs, wikis, social networking, podcasting, Web 2.0 applications, language learning in virtual worlds and interactive whiteboards (Davies et al. 2010: Section 3.7).

Warschauer (1996) and Warschauer & Healey (1998) took a different approach. Rather than focusing on the typology of CALL, they identified three historical phases of CALL, classified according to their underlying pedagogical and methodological approaches:

  • Behavioristic CALL: conceived in the 1950s and implemented in the 1960s and 1970s.
  • Communicative CALL: 1970s to 1980s.
  • Integrative CALL: embracing Multimedia and the Internet: 1990s.

Most CALL programs in Warschauer & Healey's first phase, Behavioristic CALL (1960s to 1970s), consisted of drill-and-practice materials in which the computer presented a stimulus and the learner provided a response. At first, both could be done only through text. The computer would analyse students' input and give feedback, and more sophisticated programs would react to students' mistakes by branching to help screens and remedial activities. While such programs and their underlying pedagogy still exist today, behaviouristic approaches to language learning have been rejected by most language teachers, and the increasing sophistication of computer technology has led CALL to other possibilities.

The second phase described by Warschauer & Healey, Communicative CALL, is based on the communicative approach that became prominent in the late 1970s and 1980s (Underwood 1984). In the communicative approach the focus is on using the language rather than analysis of the language, and grammar is taught implicitly rather than explicitly. It also allows for originality and flexibility in student output of language. The communicative approach coincided with the arrival of the PC, which made computing much more widely available and resulted in a boom in the development of software for language learning. The first CALL software in this phase continued to provide skill practice but not in a drill format—for example: paced reading, text reconstruction and language games—but the computer remained the tutor. In this phase, computers provided context for students to use the language, such as asking for directions to a place, and programs not designed for language learning such as Sim City, Sleuth and Where in the World is Carmen Sandiego? were used for language learning. Criticisms of this approach include using the computer in an ad hoc and disconnected manner for more marginal aims rather than the central aims of language teaching.

The third phase of CALL described by Warschauer & Healey, Integrative CALL, starting from the 1990s, tried to address criticisms of the communicative approach by integrating the teaching of language skills into tasks or projects to provide direction and coherence. It also coincided with the development of multimedia technology (providing text, graphics, sound and animation) as well as Computer-mediated communication (CMC). CALL in this period saw a definitive shift from the use of the computer for drill and tutorial purposes (the computer as a finite, authoritative base for a specific task) to a medium for extending education beyond the classroom. Multimedia CALL started with interactive laser videodiscs such as Montevidisco (Schneider & Bennion 1984) and A la rencontre de Philippe (Fuerstenberg 1993), both of which were simulations of situations where the learner played a key role. These programs later were transferred to CD-ROMs, and new role-playing games (RPGs) such as Who is Oscar Lake? made their appearance in a range of different languages.

In a later publication Warschauer changed the name of the first phase of CALL from Behavioristic CALL to Structural CALL and also revised the dates of the three phases (Warschauer 2000):

  • Structural CALL: 1970s to 1980s.
  • Communicative CALL: 1980s to 1990s.
  • Integrative CALL: 2000 onwards.

Bax (2003) took issue with Warschauer & Haley (1998) and Warschauer (2000) and proposed these three phases:

  • Restricted CALL – mainly behaviouristic: 1960s to 1980s.
  • Open CALL – i.e. open in terms of feedback given to students, software types and the role of the teacher, and including simulations and games: 1980s to 2003 (i.e. the date of Bax's article).
  • Integrated CALL – still to be achieved. Bax argued that at the time of writing language teachers were still in the Open CALL phase, as true integration could only be said to have been achieved when CALL had reached a state of "normalisation" – e.g. when using CALL was as normal as using a pen.

See also Bax & Chambers (2006) and Bax (2011), in which the topic of "normalisation" is revisited.

Flashcards

A basic use of CALL is in vocabulary acquisition using flashcards, which requires quite simple programs. Such programs often make use of spaced repetition, a technique whereby the learner is presented with the vocabulary items that need to be committed to memory at increasingly longer intervals until long-term retention is achieved. This has led to the development of a number of applications known as spaced repetition systems (SRS), including the generic Anki or SuperMemo package and programs such as BYKI and phase-6, which have been designed specifically for learners of foreign languages.

Software design and pedagogy

Above all, careful consideration must be given to pedagogy in designing CALL software, but publishers of CALL software tend to follow the latest trend, regardless of its desirability. Moreover, approaches to teaching foreign languages are constantly changing, dating back to grammar-translation, through the direct method, audio-lingualism and a variety of other approaches, to the more recent communicative approach and constructivism (Decoo 2001).

Designing and creating CALL software is an extremely demanding task, calling upon a range of skills. Major CALL development projects are usually managed by a team of people:

  • A subject specialist (also known as a content provider) – usually a language teacher – who is responsible for providing the content and pedagogical input. More than one subject specialist is required for larger CALL projects.
  • A programmer who is familiar with the chosen programming language or authoring tool.
  • A graphic designer, to produce pictures and icons, and to advise on fonts, colour, screen layout, etc.
  • A professional photographer or, at the very least, a very good amateur photographer. Graphic designers often have a background in photography too.
  • A sound engineer and a video technician will be required if the package is to contain substantial amounts of sound and video.
  • An instructional designer. Developing a CALL package is more than just putting a text book into a computer. An instructional designer will probably have a background in cognitive psychology and media technology, and will be able to advise the subject specialists in the team on the appropriate use of the chosen technology (Gimeno & Davies 2010).

CALL inherently supports learner autonomy, the final of the eight conditions that Egbert et al. (2007) cite as "Conditions for Optimal Language Learning Environments". Learner autonomy places the learner firmly in control so that he or she "decides on learning goals" (Egbert et al., 2007, p. 8).

It is all too easy when designing CALL software to take the comfortable route and produce a set of multiple-choice and gap-filling exercises, using a simple authoring tool (Bangs 2011), but CALL is much more than this; Stepp-Greany (2002), for example, describes the creation and management of an environment incorporating a constructivist and whole language philosophy. According to constructivist theory, learners are active participants in tasks in which they "construct" new knowledge derived from their prior experience. Learners also assume responsibility for their learning, and the teacher is a facilitator rather than a purveyor of knowledge. Whole language theory embraces constructivism and postulates that language learning moves from the whole to the part, rather than building sub-skills to lead towards the higher abilities of comprehension, speaking, and writing. It also emphasises that comprehending, speaking, reading, and writing skills are interrelated, reinforcing each other in complex ways. Language acquisition is, therefore, an active process in which the learner focuses on cues and meaning and makes intelligent guesses. Additional demands are placed upon teachers working in a technological environment incorporating constructivist and whole language theories. The development of teachers' professional skills must include new pedagogical as well as technical and management skills. Regarding the issue of teacher facilitation in such an environment, the teacher has a key role to play, but there could be a conflict between the aim to create an atmosphere for learner independence and the teacher's natural feelings of responsibility. In order to avoid learners' negative perceptions, Stepp-Greany points out that it is especially important for the teacher to continue to address their needs, especially those of low-ability learners.

Multimedia

Language teachers have been avid users of technology for a very long time. Gramophone records were among the first technological aids to be used by language teachers in order to present students with recordings of native speakers' voices, and broadcasts from foreign radio stations were used to make recordings on reel-to-reel tape recorders. Other examples of technological aids that have been used in the foreign language classroom include slide projectors, film-strip projectors, film projectors, videocassette recorders and DVD players. In the early 1960s, integrated courses (which were often described as multimedia courses) began to appear. Examples of such courses are Ecouter et Parler (consisting of a coursebook and tape recordings) and Deutsch durch die audiovisuelle Methode (consisting of an illustrated coursebook, tape recordings and a film-strip – based on the Structuro-Global Audio-Visual method).

During the 1970s and 1980s standard microcomputers were incapable of producing sound and they had poor graphics capability. This represented a step backwards for language teachers, who by this time had become accustomed to using a range of different media in the foreign language classroom. The arrival of the multimedia computer in the early 1990s was therefore a major breakthrough as it enabled text, images, sound and video to be combined in one device and the integration of the four basic skills of listening, speaking, reading and writing (Davies 2011: Section 1).

Examples of CALL programs for multimedia computers that were published on CD-ROM and DVD from the mid-1990s onwards are described by Davies (2010: Section 3). CALL programs are still being published on CD-ROM and DVD, but Web-based multimedia CALL has now virtually supplanted these media.

Following the arrival of multimedia CALL, multimedia language centres began to appear in educational institutions. While multimedia facilities offer many opportunities for language learning with the integration of text, images, sound and video, these opportunities have often not been fully utilised. One of the main promises of CALL is the ability to individualise learning but, as with the language labs that were introduced into educational institutions in the 1960s and 1970s, the use of the facilities of multimedia centres has often devolved into rows of students all doing the same drills (Davies 2010: Section 3.1). There is therefore a danger that multimedia centres may go the same way as the language labs. Following a boom period in the 1970s, language labs went rapidly into decline. Davies (1997: p. 28) lays the blame mainly on the failure to train teachers to use language labs, both in terms of operation and in terms of developing new methodologies, but there were other factors such as poor reliability, lack of materials and a lack of good ideas.

Managing a multimedia language centre requires not only staff who have a knowledge of foreign languages and language teaching methodology but also staff with technical know-how and budget management ability, as well as the ability to combine all these into creative ways of taking advantage of what the technology can offer. A centre manager usually needs assistants for technical support, for managing resources and even the tutoring of students. Multimedia centres lend themselves to self-study and potentially self-directed learning, but this is often misunderstood. The simple existence of a multimedia centre does not automatically lead to students learning independently. Significant investment of time is essential for materials development and creating an atmosphere conducive to self-study. Unfortunately, administrators often have the mistaken belief that buying hardware by itself will meet the needs of the centre, allocating 90% of its budget to hardware and virtually ignoring software and staff training needs (Davies et al. 2011: Foreword). Self-access language learning centres or independent learning centres have emerged partially independently and partially in response to these issues. In self-access learning, the focus is on developing learner autonomy through varying degrees of self-directed learning, as opposed to (or as a complement to) classroom learning. In many centres learners access materials and manage their learning independently, but they also have access to staff for help. Many self-access centres are heavy users of technology and an increasing number of them are now offering online self-access learning opportunities. Some centres have developed novel ways of supporting language learning outside the context of the language classroom (also called 'language support') by developing software to monitor students' self-directed learning and by offering online support from teachers. Centre managers and support staff may need to have new roles defined for them to support students' efforts at self-directed learning: v. Mozzon-McPherson & Vismans (2001), who refer to a new job description, namely that of the "language adviser".

Internet

The emergence of the World Wide Web (now known simply as "the Web") in the early 1990s marked a significant change in the use of communications technology for all computer users. Email and other forms of electronic communication had been in existence for many years, but the launch of Mosaic, the first graphical Web browser, in 1993 brought about a radical change in the ways in which we communicate electronically. The launch of the Web in the public arena immediately began to attract the attention of language teachers. Many language teachers were already familiar with the concept of hypertext on stand-alone computers, which made it possible to set up non-sequential structured reading activities for language learners in which they could point to items of text or images on a page displayed on the computer screen and branch to any other pages, e.g. in a so-called "stack" as implemented in the HyperCard program on Apple Mac computers. The Web took this one stage further by creating a worldwide hypertext system that enabled the user to branch to different pages on computers anywhere in the world simply by pointing and clicking at a piece of text or an image. This opened up access to thousands of authentic foreign-language websites to teachers and students that could be used in a variety of ways. A problem that arose, however, was that this could lead to a good deal of time-wasting if Web browsing was used in an unstructured way (Davies 1997: pp. 42–43), and language teachers responded by developing more structured activities and online exercises (Leloup & Ponterio 2003). Davies (2010) lists over 500 websites, where links to online exercises can be found, along with links to online dictionaries and encyclopaedias, concordancers, translation aids and other miscellaneous resources of interest to the language teacher and learner.

The launch of the (free) Hot Potatoes (Holmes & Arneil) authoring tool, which was first demonstrated publicly at the EUROCALL 1998 conference, made it possible for language teachers to create their own online interactive exercises. Other useful tools are produced by the same authors.

In its early days the Web could not compete seriously with multimedia CALL on CD-ROM and DVD. Sound and video quality was often poor, and interaction was slow. But now the Web has caught up. Sound and video are of high quality and interaction has improved tremendously, although this does depend on sufficient bandwidth being available, which is not always the case, especially in remote rural areas and developing countries. One area in which CD-ROMs and DVDs are still superior is in the presentation of listen/respond/playback activities, although such activities on the Web are continually improving.

Since the early 2000s there has been a boom in the development of so-called Web 2.0 applications. Contrary to popular opinion, Web 2.0 is not a new version of the Web, rather it implies a shift in emphasis from Web browsing, which is essentially a one-way process (from the Web to the end-user), to making use of Web applications in the same way as one uses applications on a desktop computer. It also implies more interaction and sharing. Walker, Davies & Hewer (2011: Section 2.1) list the following examples of Web 2.0 applications that language teachers are using:

There is no doubt that the Web has proved to be a main focus for language teachers, who are making increasingly imaginative use of its wide range of facilities: see Dudeney (2007) and Thomas (2008). Above all, the use of Web 2.0 tools calls for a careful reexamination of the role of the teacher in the classroom (Richardson 2006).

Corpora and concordancers

Corpora have been used for many years as the basis of linguistic research and also for the compilation of dictionaries and reference works such as the Collins Cobuild series, published by HarperCollins. Tribble & Barlow (2001), Sinclair (2004) and McEnery & Wilson (2011) describe a variety of ways in which corpora can be used in language teaching.

An early reference to the use of electronic concordancers in language teaching can be found in Higgins & Johns (1984: pp. 88–94), and many examples of their practical use in the classroom are described by Lamy & Klarskov Mortensen (2010).

It was Tim Johns (1991), however, who raised the profile of the use of concordancers in the language classroom with his concept of Data-driven learning (DDL). DDL encourages learners to work out their own rules about the meaning of words and their usage by using a concordancer to locate examples in a corpus of authentic texts. It is also possible for the teacher to use a concordancer to find examples of authentic usage to demonstrate a point of grammar or typical collocations, and to generate exercises based on the examples found. Various types of concordancers and where they can be obtained are described by Lamy & Klarskov Mortensen (2011).

Robb (2003) shows how it is possible to use Google as a concordancer, but he also points out a number of drawbacks, for instance there is no control over the educational level, nationality, or other characteristics of the creators of the texts that are found, and the presentation of the examples is not as easy to read as the output of a dedicated concordancer that places the key words (i.e. the search terms) in context.

Virtual worlds

Virtual worlds date back to the adventure games and simulations of the 1970s, for example Colossal Cave Adventure, a text-only simulation in which the user communicated with the computer by typing commands at the keyboard. Language teachers discovered that it was possible to exploit these text-only programs by using them as the basis for discussion. Jones G. (1986) describes an experiment based on the Kingdom simulation, in which learners played roles as members of a council governing an imaginary kingdom. A single computer in the classroom was used to provide the stimulus for discussion, namely simulating events taking place in the kingdom: crop planting time, harvest time, unforeseen catastrophes, etc.

The early adventure games and simulations led on to multi-user variants, which were known as MUDs (Multi-user domains). Like their predecessors, MUDs were text-only, with the difference that they were available to a wider online audience. MUDs then led on to MOOs (Multi-user domains object-oriented), which language teachers were able to exploit for teaching foreign languages and intercultural understanding: see Donaldson & Kötter (1999) and (Shield 2003).

The next major breakthrough in the history of virtual worlds was the graphical user interface. Lucasfilm's Habitat (1986), was one of the first virtual worlds that was graphically based, albeit only in a two-dimensional environment. Each participant was represented by a visual avatar who could interact with other avatars using text chat.

Three-dimensional virtual worlds such as Traveler and Active Worlds, both of which appeared in the 1990s, were the next important development. Traveler included the possibility of audio communication (but not text chat) between avatars who were represented as disembodied heads in a three-dimensional abstract landscape. Svensson (2003) describes the Virtual Wedding Project, in which advanced students of English made use of Active Worlds as an arena for constructivist learning.

The 3D world of Second Life was launched in 2003. Initially perceived as another role-playing game (RPG), it began to attract the interest of language teachers with the launch of the first of the series of SLanguages conferences in 2007. Walker, Davies & Hewer (2011: Section 14.2.1) and Molka-Danielsen & Deutschmann (2010) describe a number of experiments and projects that focus on language learning in Second Life. See also the Misplaced Pages article Virtual world language learning.

To what extent Second Life and other virtual worlds will become established as important tools for teachers of foreign languages remains to be seen. It has been argued by Dudeney (2010) in his That's Life blog that Second Life is "too demanding and too unreliable for most educators". The subsequent discussion shows that this view is shared by many teachers, but many others completely disagree.

Regardless of the pros and cons of Second Life, language teachers' interest in virtual worlds continues to grow. The joint EUROCALL/CALICO Virtual Worlds Special Interest Group was set up in 2009, and there are now many areas in Second Life that are dedicated to language learning and teaching, for example the commercial area for learners of English, which is managed by Language Lab, and free areas such as the region maintained by the Goethe-Institut and the EduNation Islands. There are also examples of simulations created specifically for language education, such as those produced by the EC-funded NIFLAR and AVALON projects. NIFLAR is implemented both in Second Life and in Opensim.

Human language technologies

Human language technologies (HLT) comprise a number of areas of research and development that focus on the use of technology to facilitate communication in a multilingual information society. Human language technologies are areas of activity in departments of the European Commission that were formerly grouped under the heading language engineering (Gupta & Schulze 2011: Section 1.1).

The parts of HLT that is of greatest interest to the language teacher is natural language processing (NLP), especially parsing, as well as the areas of speech synthesis and speech recognition.

Speech synthesis has improved immeasurably in recent years. It is often used in electronic dictionaries to enable learners to find out how words are pronounced. At word level, speech synthesis is quite effective, the artificial voice often closely resembling a human voice. At phrase level and sentence level, however, there are often problems of intonation, resulting in speech production that sounds unnatural even though it may be intelligible. Speech synthesis as embodied in text to speech (TTS) applications is invaluable as a tool for unsighted or partially sighted people. Gupta & Schulze (2010: Section 4.1) list several examples of speech synthesis applications.

Speech recognition is less advanced than speech synthesis. It has been used in a number of CALL programs, in which it is usually described as automatic speech recognition (ASR). ASR is not easy to implement. Ehsani & Knodt (1998) summarise the core problem as follows:

"Complex cognitive processes account for the human ability to associate acoustic signals with meanings and intentions. For a computer, on the other hand, speech is essentially a series of digital values. However, despite these differences, the core problem of speech recognition is the same for both humans and machines: namely, of finding the best match between a given speech sound and its corresponding word string. Automatic speech recognition technology attempts to simulate and optimize this process computationally."

Programs embodying ASR normally provide a native speaker model that the learner is requested to imitate, but the matching process is not 100% reliable and may result in a learner's perfectly intelligible attempt to pronounce a word or phrase being rejected (Davies 2010: Section 3.4.6 and Section 3.4.7).

Parsing is used in a number of ways in CALL. Gupta & Schulze (2010: Section 5) describe how parsing may be used to analyse sentences, presenting the learner with a tree diagram that labels the constituent parts of speech of a sentence and shows the learner how the sentence is structured.

Parsing is also used in CALL programs to analyse the learner's input and diagnose errors. Davies (2002) writes:

"Discrete error analysis and feedback were a common feature of traditional CALL, and the more sophisticated programs would attempt to analyse the learner's response, pinpoint errors, and branch to help and remedial activities. ... Error analysis in CALL is, however, a matter of controversy. Practitioners who come into CALL via the disciplines of computational linguistics, e.g. Natural Language Processing (NLP) and Human Language Technologies (HLT), tend to be more optimistic about the potential of error analysis by computer than those who come into CALL via language teaching. An alternative approach is the use of Artificial Intelligence (AI) techniques to parse the learner's response – so-called intelligent CALL (ICALL) – but there is a gulf between those who favour the use of AI to develop CALL programs (Matthews 1994) and, at the other extreme, those who perceive this approach as a threat to humanity (Last 1989:153)".

Underwood (1989) and Heift & Schulze (2007) present a more positive picture of AI.

Research into speech synthesis, speech recognition and parsing and how these areas of NLP can be used in CALL are the main focus of the NLP Special Interest Group within the EUROCALL professional association and the ICALL Special Interest Group within the CALICO professional association. The EUROCALL NLP SIG also maintains a Ning.

Impact

The question of the impact of CALL in language learning and teaching has been raised at regular intervals ever since computers first appeared in educational institutions (Davies & Hewer 2011: Section 3). Recent large-scale impact studies include the study edited by Fitzpatrick & Davies (2003) and the EACEA (2009) study, both of which were produced for the European Commission.

A distinction needs to be made between the impact and the effectiveness of CALL. Impact may be measured quantitatively and qualitatively in terms of the uptake and use of ICT in teaching foreign languages, issues of availability of hardware and software, budgetary considerations, Internet access, teachers' and learners' attitudes to the use of CALL, changes in the ways in which languages are learnt and taught, and paradigm shifts in teachers' and learners' roles. Effectiveness, on the other hand, usually focuses on assessing to what extent ICT is a more effective way of teaching foreign languages compared to using traditional methods – and this is more problematic as so many variables come into play. Worldwide, the picture of the impact of CALL is extremely varied. Most developed nations work comfortably with the new technologies, but developing nations are often beset with problems of costs and broadband connectivity. Evidence on the effectiveness of CALL – as with the impact of CALL – is extremely varied and many research questions still need to be addressed and answered. Hubbard (2002) presents the results of a CALL research survey that was sent to 120 CALL professionals from around the world asking them to articulate a CALL research question they would like to see answered. Some of the questions have been answered but many more remain open. Leakey (2011) offers an overview of current and past research in CALL and proposes a comprehensive model for evaluating the effectiveness of CALL platforms, programs and pedagogy.

A crucial issue is the extent to which the computer is perceived as taking over the teacher's role. Warschauer (1996: p. 6) perceived the computer as playing an "intelligent" role, and claimed that a computer program "should ideally be able to understand a user's spoken input and evaluate it not just for correctness but also for appropriateness. It should be able to diagnose a student's problems with pronunciation, syntax, or usage and then intelligently decide among a range of options (e.g. repeating, paraphrasing, slowing down, correcting, or directing the student to background explanations)." Jones C. (1986), on the other hand, rejected the idea of the computer being "some kind of inferior teacher-substitute" and proposed a methodology that focused more on what teachers could do with computer programs rather than what computer programs could do on their own: "in other words, treating the computer as they would any other classroom aid". Warschauer's high expectations in 1996 have still not been fulfilled, and currently there is an increasing tendency for teachers to go down the route proposed by Jones, making use of a variety of new tools such as corpora and concordancers, interactive whiteboards and applications for online communication.

Since the advent of the Web there has been an explosion in online learning, but to what extent it is effective is open to criticism. Felix (2003) takes a critical look at popular myths attached to online learning from three perspectives, namely administrators, teachers and students. She concludes: "That costs can be saved in this ambitious enterprise is clearly a myth, as are expectations of saving time or replacing staff with machines."

As for the effectiveness of CALL in promoting the four skills, Felix (2008) claims that there is "enough data in CALL to suggest positive effects on spelling, reading and writing", but more research is needed in order to determine its effectiveness in other areas, especially speaking online. She claims that students' perceptions of CALL are positive, but she qualifies this claim by stating that the technologies need to be stable and well supported, drawing attention to concerns that technical problems may interfere with the learning process. She also points out that older students may not feel comfortable with computers and younger students may not possess the necessary meta-skills for coping effectively in the challenging new environments. Training in computer literacy for both students and teachers is essential, and time constraints may pose additional problems. In order to achieve meaningful results she recommends "time-series analysis in which the same group of students is involved in experimental and control treatment for a certain amount of time and then switched – more than once if possible".

Types of technology training in CALL for language teaching professionals certainly vary. Within second language teacher education programs, namely pre-service course work, we can find "online courses along with face-to-face courses", computer technology incorporated into a more general second language education course, "technology workshops","a series of courses offered throughout the teacher education programs, and even courses specifically designed for a CALL certificate and a CALL graduate degree" The Organization for Economic Cooperation and Development has identified four levels of courses with only components, namely "web-supplemented, web-dependent, mixed mod and fully online".

There is a rapidly growing interest in resources about the use of technology to deliver CALL. Journals that have issues that "deal with how teacher education programs help prepare language teachers to use technology in their own classrooms" include Language Learning and Technology (2002), Innovations in Language Learning and Teaching (2009) and the TESOL international professional association's publication of technology standards for TESOL includes a chapter on preparation of teacher candidates in technology use, as well as the upgrading of teacher educators to be able to provide such instruction. Both CALICO and EUROCALL have special interest groups for teacher education in CALL.

Professional associations

The following professional associations are dedicated to the promulgation of research, development and practice relating to the use of new technologies in language learning and teaching. Most of them organise conferences and publish journals on CALL.

  • APACALL: The Asia-Pacific Association for Computer-Assisted Language Learning. APACALL publishes the APACALL Book Series and APACALL Newsletter Series.
  • AsiaCALL: The Asia Association of Computer Assisted Language Learning, Korea. AsiaCALL publishes the AsiaCALL Online Journal.
  • Association of University Language Centres (AULC) in the UK and Ireland.
  • CALICO: Established in 1982. Currently based at Texas State University, USA. CALICO publishes the CALICO Journal.
  • EUROCALL: Founded by a group of enthusiasts in 1986 and established with the aid of European Commission funding as a formal professional association in 1993. Currently based at the University of Ulster, Northern Ireland. EUROCALL's journal, ReCALL, is published by Cambridge University Press. EUROCALL also publishes the EUROCALL Review.
  • IALLT: The US-based International Association for Language Learning Technology, originally known as IALL (International Association for Learning Labs). IALLT is a professional organisation dedicated to promoting effective uses of media centres for language teaching, learning, and research. IALLT published the IALLT Journal until 2018. In early 2019, IALLT officially merged the journal into The FLTMAG .
  • IATEFL: The UK-based International Association of Teachers of English as a Foreign Language. IATEFL embraces the Learning Technologies Special Interest Group (LTSIG) and publishes the CALL Review newsletter.
  • JALTCALL: Japan. The JALT CALL SIG publishes The JALT CALL Journal.
  • IndiaCALL:The India Association of Computer Assisted Language Learning. IndiaCALL is an affiliate of AsiaCALL, an associate of IATEFL, and an IALLT Regional Group.
  • LET: The Japan Association for Language Education and Technology (LET), formerly known as the Language Laboratory Association (LLA), and now embraces a wider range of language learning technologies.
  • PacCALL: The Pacific Association for Computer Assisted Language Learning, promoting CALL in the Pacific, from East to Southeast Asia, Oceania, across to the Americas. Organises the Globalization and Localization in Computer-Assisted Language Learning (GLoCALL) conference jointly with APACALL.
  • TCLT: Technology and Chinese Language Teaching, an organization of Chinese CALL studies in the United States, with biennial conference and workshops since 2000 and a double blind, peer-reviewed online publication-Journal of Technology and Chinese Language Teaching since 2010 and in-print supplement Series of Technology and Chinese Language Teaching in the U.S. with China Social Sciences Press since 2012.
  • WorldCALL: A worldwide umbrella association of CALL associations. The first WorldCALL conference was held at the University of Melbourne in 1998. The second WorldCALL conference took place in Banff, Canada, 2003. The third WorldCALL took place in Japan in 2008. The fourth WorldCALL conference took place in Glasgow, Scotland, 2013. The fifth WorldCALL conference took place in Concepción, Chile in 2018.

See also

References

  1. Higgins, John (1983). "Computer assisted language learning". Language Teaching. 16 (2): 102–114. doi:10.1017/S0261444800009988. S2CID 145169394.
  2. ^ Levy, M. (1997). CALL: Context and Conceptualisation. Oxford: Oxford University Press.
  3. ^ Schmid, Euline Cutrim. (2009). Interactive Whiteboard Technology in the Language Classroom: Exploring New Pedagogical Opportunities. Saarbrücken, Germany: VDM Verlag Dr. Müller.
  4. ^ Lamy M.-N. & Hampel R. (2007) Online communication in language learning and teaching, Houndmills: Palgrave Macmillan.
  5. Shield, L., & Kukulska-Hulme, A. (Eds.). (2008). Special edition of ReCALL (20, 3) on Mobile Assisted Language Learning.
  6. Davies G. & Higgins J. (1982) Computers, language and language learning, London: CILT.
  7. Bush M. & Terry R. (1997) (eds.) Technology-enhanced language learning, Lincolnwood, Illinois: National Textbook Company.
  8. Pegrum M. (2009) From blogs to bombs: The future of digital technologies in education, Perth: University of Western Australia Press.
  9. ^ Davies G., Walker R., Rendall H. & Hewer S. (2011) Introduction to Computer Assisted Language Learning (CALL). Module 1.4 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod1-4.htm
  10. Levy, M.; Hubbard, P. (2005). "Why call CALL "CALL"?". Computer Assisted Language Learning. 18 (3): 143–149. doi:10.1080/09588220500208884. S2CID 62662197.
  11. Marty, F (1981). "Reflections on the use of computers in second language acquisition". System. 9 (2): 85–98. doi:10.1016/0346-251x(81)90023-3.
  12. "Reviews of Language Courses". Lang1234. Retrieved 12 September 2012.
  13. Sanders R. (ed.) (1995) Thirty years of computer-assisted language instruction, Festschrift for John R. Russell, CALICO Journal Special Issue, 12, 4.
  14. Delcloque P. (2000) History of CALL : http://www.ict4lt.org/en/History_of_CALL.pdf
  15. Davies G. (2005) Computer Assisted Language Learning: Where are we now and where are we going? : http://www.camsoftpartners.co.uk/docs/UCALL_Keynote.htm
  16. Hubbard P. (2009) (ed.) Computer-assisted language learning, Volumes I-IV, Routledge: London and New York: http://www.stanford.edu/~efs/callcc/
  17. Language Learning & Technology (2010) 14, 3, pp. 14–18 : http://llt.msu.edu/issues/october2010/index.html
  18. Butler-Pascoe M. E. (2011) "The history of CALL: the intertwining paths of technology and second/foreign language teaching", International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT) 1, 1: 16–32: http://www.igi-global.com/ijcallt
  19. Davies G. & Higgins J. (1985) Using computers in language learning: a teacher's guide, London: CILT.
  20. Jones C. & Fortescue S. (1987) Using computers in the language classroom, Harlow: Longman.
  21. Hardisty D. & Windeatt S. (1989) CALL, Oxford: Oxford University Press.
  22. ^ Warschauer M. (1996) "Computer-assisted language learning: an introduction". In Fotos S. (ed.) Multimedia language teaching, Tokyo: Logos International : http://www.ict4lt.org/en/warschauer.htm
  23. Warschauer, M.; Healey, D. (1998). "Computers and language learning: an overview". Language Teaching. 31 (2): 57–71. doi:10.1017/s0261444800012970. S2CID 12813227.
  24. Underwood J. (1984) Linguistics, computers and the language teacher: a communicative approach, Rowley, Massachusetts: Newbury House.
  25. Schneider E.W. & Bennion J.L. (1984) "Veni, vidi, vici, via videodisc: a simulator for instructional courseware". In Wyatt D.H. (ed.) Computer-assisted language instruction, Oxford: Pergamon.
  26. Fuerstenberg G. (1993) A la rencontre de Philippe: Videodisc, Software, Teacher's Manual and Student Activities Workbook: Yale University Press : http://web.mit.edu/fll/www/projects/Philippe.html
  27. Warschauer M. (2000) "CALL for the 21st Century", IATEFL and ESADE Conference, 2 July 2000, Barcelona, Spain.
  28. Bax, S (2003). "CALL – past, present and future". System. 31 (1): 13–28. doi:10.1016/s0346-251x(02)00071-4.
  29. Bax, S.; Chambers, A. (2006). "Making CALL work: towards normalisation". System. 34 (4): 465–479. doi:10.1016/j.system.2006.08.001.
  30. Bax S. (2011) "Normalisation revisited: the effective use of technology in language education", International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT) 1, 2: 1–15: http://www.igi-global.com/ijcallt
  31. Mounteney M. Spaced repetition learning systems (SRS): http://www.omniglot.com/language/srs.php. Accessed 14 July 2011.
  32. BYKI (Before You Know It): A personalized language-learning system, designed to promote language proficiency by expanding the learner's vocabulary. The system makes use of a flashcard interface and tracks everything that has been learned, automatically presenting and re-presenting the material at appropriate intervals: http://www.byki.com/
  33. phase-6: A flashcard system that presents vocabulary at phased intervals, aiding long-term retention. http://www.phase-6.com/
  34. Decoo W. (2001) On the mortality of language learning methods. Paper given as the James L. Barker lecture on 8 November 2001 at Brigham Young University : https://web.archive.org/web/20080208190123/http://webh01.ua.ac.be/didascalia/mortality.htm
  35. Gimeno-Sanz A. & Davies G. (2010) CALL software design and implementation. Module 3.2 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod3-2.htm
  36. Egbert J., Chao C.-C., & Hanson-Smith E. (2007) Introduction: Foundations for Teaching and Learning. In Egbert J. & E. Hanson-Smith (eds.) CALL environments: Research, practice, and critical issues (2nd edition). Alexandria, VA: TESOL. (pp. 1–14).
  37. Bangs P. (2011) Introduction to CALL authoring programs. Module 2.5 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod2-5.htm
  38. Stepp-Greany J. (2002) "Student perceptions on language learning in a technological environment: implications for the new millennium", Language Learning & Technology 6, 1: 165–180 : http://llt.msu.edu/vol6num1/steppgreany/default.html
  39. Côté D., Narins Levy S. & O'Connor P. (1962) Ecouter et Parler, Paris: Librairie Marcel Didier.
  40. Burgdorf I. Montani K., Z. Skreb Z. & Vidovic M. (1962) Deutsch durch die audio-visuelle Methode, New York: Holt Rinehart and Winston 1962.
  41. ^ Davies G. (2011) Introduction to multimedia CALL. Module 2.2 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod2-2.htm
  42. ^ Davies G. (1997) "Lessons from the past, lessons for the future: 20 years of CALL". In Korsvold A-K. & Rüschoff B. (eds.) New technologies in language learning and teaching, Strasbourg: Council of Europe, p. 28. Also on the Web in a revised edition (2009): http://www.camsoftpartners.co.uk/coegdd1.htm/ Archived 16 November 2010 at the Wayback Machine
  43. Davies G., Hamilton R., Weidmann B., Gabel S., Legenhausen L., Meus V. & Myers S. (2011) Managing a multimedia language centre. Module 3.1 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod3-1.htm
  44. Mozzon-McPherson M. & Vismans R. (eds.) (2001) Beyond language teaching towards language advising, London: CILT.
  45. LeLoup J. & Ponterio R. (2003) "Interactive and multimedia techniques in online language lessons: a sampler", Language Learning & Technology 7, 3 : http://llt.msu.edu/vol7num3/net/default.html
  46. Davies G. (2010) Graham Davies's Favourite Websites
  47. Holmes M. & Arneil S. Hot Potatoes, University of Victoria, Canada. Downloadable from: http://hotpot.uvic.ca: A library of Clipart for use with Hot Potatoes and other authoring tools is also available at the University of Victoria site: http://hcmc.uvic.ca/clipart/
  48. ^ Walker R., Davies G. & Hewer S. (2011) Introduction to the Internet. Module 1.5 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod1-5.htm
  49. Ward, Jason M. (2004). "Blog Assisted Language Learning (BALL): Push button publishing for the pupils" (PDF). TEFL Web Journal. 3 (1). Archived from the original (PDF) on 9 December 2023.
  50. Dudeney G. (2007) The Internet and the language classroom, Second Edition, Cambridge: Cambridge University Press.
  51. Thomas M. (2008) Handbook of research on Web 2.0 and second language learning, Hershey, Pennsylvania, USA: IGI Global.
  52. Richardson W. (2006) Blogs, Wikis, Podcasts and the Powerful Web Tools for Classrooms. Thousand Oaks: Corwin Press.
  53. HarperCollins: http://www.harpercollins.co.uk
  54. Tribble C. & Barlow M. (eds.) (2001) Using corpora in language teaching and learning, Special Issue, Language Learning & Technology 5, 3 : http://llt.msu.edu/vol5num3/default.html
  55. Sinclair J. (ed.) (2004) How to use corpora in language teaching, Amsterdam: John Benjamins
  56. McEnery A. & Wilson A. (2011) Corpus linguistics. Module 3.4 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod3-4.htm
  57. Higgins J. & Johns T. (1984) Computers in language learning, London: Collins.
  58. ^ Lamy M-N. & Klarskov Mortensen H. J. (2011) Using concordance programs in the Modern Foreign Languages classroom. Module 2.4 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod2-4.htm
  59. Johns T. (1991) "From printout to handout: grammar and vocabulary teaching in the context of Data Driven Learning". In Johns T. & King P. (eds.) Classroom concordancing, Special issue of ELR Journal 4, University of Birmingham, Centre for English Language Studies: 27–45.
  60. Robb T. (2003) "Google as a Quick 'n Dirty Corpus Tool", TESL-EJ 7, 2 : http://tesl-ej.org/ej26/int.html
  61. Jones, G (1986). "Computer simulations in language teaching – the KINGDOM experiment". System. 14 (2): 179–186. doi:10.1016/0346-251x(86)90007-2.
  62. Donaldson, R.P.; Kötter, M. (1999). "Language learning in cyberspace: teleporting the classroom into the target culture". CALICO Journal. 16 (4): 531–558. doi:10.1558/cj.v16i4.531-557.
  63. Shield L. (2003) "MOO as a language learning tool". In Felix U. (ed.) Language learning online: towards best practice, Lisse: Swets & Zeitlinger.
  64. Svensson P. (2003) "Virtual worlds as arenas for language learning". In Felix U. (ed.) Language learning online: towards best practice, Lisse: Swets & Zeitlinger.
  65. SLanguages: http://www.slanguages.net/home.php Archived 13 January 2011 at the Wayback Machine
  66. Molka-Danielsen, J. & Deutschmann, M. (eds.) (2009) Learning and teaching in the virtual world of Second Life, Trondheim, Norway: Tapir Academic Press.
  67. Second Life – The Long Goodbye: http://slife.dudeney.com/?p=446
  68. EUROCALL/CALICO Virtual Worlds Special Interest Group: http://virtualworldssig.ning.com
  69. LanguageLab: http://www.languagelab.com/en/
  70. Goethe-Institut: http://www.goethe.de/frm/sec/enindex.htm
  71. EduNation Islands: http://edunation-islands.wikispaces.com
  72. Networked Interaction in Foreign Language Acquisition and Research (NIFLAR): http://niflar.ning.com Archived 30 September 2010 at the Wayback Machine
  73. Access to Virtual and Action Learning live ONline (AVALON): http://avalon-project.ning.com/ Archived 14 July 2011 at the Wayback Machine
  74. ^ Gupta P. & Schulze M. (2011) Human Language Technologies (HLT). Module 3.5 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod3-5.htm
  75. Ehsani F. & Knodt E. (1998) "Speech technology in computer-aided language learning: strengths and limitations of a new CALL paradigm", Language Learning and Technology 2, 1: 45–60 : http://llt.msu.edu/vol2num1/article3/index.html
  76. Davies G. (2002) Article on CALL in the Good Practice Guide at the website of the Subject Centre for Languages, Linguistics and Area Studies (LLAS), University of Southampton : http://www.llas.ac.uk/resources/gpg/61
  77. Matthews, C (1994). "Intelligent Computer Assisted Language Learning as cognitive science: the choice of syntactic frameworks for language tutoring". Journal of Artificial Intelligence in Education. 5 (4): 533–56.
  78. Last R.W. (1989) Artificial intelligence techniques in language learning, Chichester: Ellis Horwood.
  79. Underwood, J (1989). "On the edge: Intelligent CALL in the 1990s". Computers and the Humanities. 23: 71–84. doi:10.1007/bf00058770. S2CID 60043026.
  80. Heift T. & Schulze M. (eds.) (2007) Errors and intelligence in CALL: parsers and pedagogues, New York: Routledge.
  81. EUROCALL NLP Special Interest Group: http://siglp.eurocall-languages.org/
  82. CALICO ICALL Special Interest Group:
  83. EUROCALL NLP Special Interest Group Ning: http://nlpsig.ning.com/ Archived 14 July 2011 at the Wayback Machine
  84. Davies G. & Hewer S. (2011) Introduction to new technologies and how they can contribute to language learning and teaching. Module 1.1 in Davies G. (ed.) Information and Communications Technology for Language Teachers (ICT4LT), Slough, Thames Valley University : http://www.ict4lt.org/en/en_mod1-1.htm
  85. Fitzpatrick A. & Davies G. (eds.) (2003) The impact of Information and Communications Technologies on the teaching of foreign languages and on the role of teachers of foreign languages, EC Directorate General of Education and Culture.
  86. Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission (2009) Study on the impact of ICT and new media on language learning : http://eacea.ec.europa.eu/llp/studies/study_impact_ict_new_media_language_learning_en.php
  87. mahmoudi, elham (30 June 2012). "Attitude and student's performance in Computer Assisted English Language Learning (CALL) for Learning Vocabulary". Procedia – Social and Behavioral Sciences. 66: 489–498. doi:10.1016/j.sbspro.2012.11.293.
  88. Hubbard P. (2002) Survey of unanswered questions in Computer Assisted Language Learning: Effectiveness issues : http://www.stanford.edu/~efs/callsurvey/index.html
  89. Leakey J. (2011) Evaluating Computer Assisted Language Learning: an integrated approach to effectiveness research in CALL, Bern: Peter Lang.
  90. Jones, C (1986). "It's not so much the program: more what you do with it: the importance of methodology in CALL". System. 14 (2): 171–178. doi:10.1016/0346-251x(86)90006-0.
  91. Felix U. (2003) "Teaching languages online: deconstructing the myths", Australian Journal of Educational Technology 19, 1: 118–138 : http://www.ascilite.org.au/ajet/ajet19/felix.html Archived 14 March 2011 at the Wayback Machine
  92. Felix, U (2008). "The unreasonable effectiveness of CALL: what have we learned in two decades of research?". ReCALL. 20 (2): 141–161. doi:10.1017/s0958344008000323. S2CID 26766962.
  93. Hong, K. H. (2010). "CALL teacher education as an impetus for 12 teachers in integrating technology". ReCALL. 22 (1): 53–69. doi:10.1017/s095834400999019X. S2CID 27700801.
  94. Murray, D. E. (2013) A Case for Online English Language Teacher Education. The International Research Foundation for English Language Education. http://www.tirfonline.org/wp-content/uploads/2013/04/TIRF_OLTE_One-PageSpread_2013.pdf
  95. Murray, D. E. (2013) A Case for Online English Language Teacher Education. The International Research Foundation for English Language Education. http://www.tirfonline.org/wp-content/uploads/2013/04/TIRF_OLTE_One-PageSpread_2013
  96. Links to these associations can be found in the Resource Centre of the ICT4LT website: http://www.ict4lt.org/en/en_resource.htm#profassocs

Hong, K. H. (2010) CALL teacher education as an impetus for 12 teachers in integrating technology. ReCALL, 22 (1), 53–69. doi:10.1017/s095834400999019X

Murray, D. E. (2013) A Case for Online English Language Teacher Education. The International Research Foundation for English Language Education. http://www.tirfonline.org/wp-content/uploads/2013/04/TIRF_OLTE_One-PageSpread_2013.pdf

Further reading

  • Chapelle, C.A., & Sauro, S. (Eds.). (2017).The handbook of technology and second language teaching and learning. Oxford: Wiley-Blackwell.
  • Son, J.-B. (2018). Teacher development in technology-enhanced language teaching. London: Palgrave Macmillan.

External links

Categories: