Hypermedia as an educational technology: a review of the quantitative research literature on learner comprehension, control and style

Andrew Dillon and Ralph Gabbard

This item is not the definitive copy. Please use the following citation when referencing this material: Dillon and Gabbard (1998) Hypermedia as an educational technology: a review of the empirical literature on learner comprehension, control and style. Review of Educational Research, 68(3), 322, 349. Reprinted in P. Smith and A. Pellegrinni (eds.) (2000) The Psychology of Education: Major Themes, London: Routledge, 3, 496-531.

Abstract

By virtue of its enabling rapid, non-linear access to multiple forms of information, hypermedia technology is considered a major advance in the development of educational tools to enhance learning and a massive literature on the use of hypermedia in education has emerged. The present review examines the published findings from experimental studies of hypermedia which emphasized quantitative, empirical methods to assess learning outcomes. Specifically, the review categorizes this research into three themes: studies of learner comprehension compared across hypermedia and other media; effects on learning outcome offered by increased learner control in hypermedia environments, and the individual differences that exist in learner response to hypermedia. The review concludes that to date, the benefits of hypermedia in education are limited to learning tasks reliant on repeated manipulation and searching of information, and are differentially distributed across learners depending on their ability and preferred learning style. Methodological and analytical shortcomings in this literature limit the generalizability of all findings in this domain. Suggestions for addressing these problems in future research and theory development are outlined.

Introduction

In recent years the emergence of digital documents has progressed from word processed text, through standalone hypermedia, to the World Wide Web. With each new stage of technological development, the lessons learned from user studies of previous technologies tend to be overlooked. With one eye on the future, many educators and literary scholars are predicting nothing less than a paradigm shift in the manner in which we understand the learning experience and the education process as a result of hypermedia technologies in general, and the World Wide Web in particular. For example, Landow (1992) writes:

"Electronic linking shifts the boundaries between one text and another as well as between the author and the reader and between the teacher and the student" [Landow, 1992, 33].

In a similar vein, Dryden (1994) argues that:

"hypermedia environments can indeed promote the appreciation of literature (and of texts in other disciplines) as they nurture the growth of the learner in intellect and spirit." [284] (Furthermore) "hypermedia has the potential to transform the structure of both classrooms and entire institutions -schools and universities- and to make the teaching and practice of literate thinking and behavior a truly democratic enterprise that respects and serves the needs of both the individual learner and the larger community of learners." [284]

These are but two quotations from a large and expanding literature on hypermedia where this technology is unquestioningly advocated as an advance in educational technology for one or more of the following reasons:

(1) hypermedia enables non-linear access to vast amounts of information (Nielsen 1995)
(2) users can explore information in-depth on demand (Collier, 1987)
(3) interaction with the instructional material can be self-paced (Barratt, 1988)
(4) hypermedia is attention-capturing or engaging to use (Jonassen, 1989)
(5) hypermedia represents a natural form of representation with respect to the workings of the human mind (Delany and Gilbert, 1991)

Such writing is strong on claims, but so far short on supporting evidence from studies of learners as some researchers have continually noted. For example, McKnight et al (1991 and 1996) argued that the empirical evidence for any educational benefits of hypermedia was not convincing, and recent critiques (e.g., Dillon 1996) suggest that the breakthroughs promised by hypermedia advocates are more mythical than real. Most telling perhaps, Landauer (1995) reported that despite numerous published reports on the topic of hypermedia use, he could only find nine studies of human performance with this technology that met even minimally acceptable scientific criteria. Chen and Rada (1996) managed to identify 23 experimental studies involving human interaction with various forms of hypertext up to 1993, adopting less strict criteria for acceptance, and counting papers with more than one study repeatedly on the basis of number of experiments reported (they identified a total of 18 papers), but even their analysis of effect size showed little real advantage for hypertext over other media in general information tasks.

Focus of the review
In an attempt to advance the arguments about educational hypermedia onto an empirical footing, the present paper extends Landauer's analysis into the learning domain and seeks to provide a baseline review of the experimental findings to date on the quantitative effects of hypertext/hypermedia on learning outcome. Following Landauer's initiative we sought published studies of hypermedia use and learning outcome that were empirical (based on user data), experimental (here considered as meeting rudimentary scientific requirements for selection, manipulation and control of variables), and primarily quantitative (though use of qualitative data in parallel did not rule out published studies). Our emphasis was on the measured effects of hypermedia usage on learning outcomes, which we defined here as any desirable and demonstrable changes in learner behavior or task performance as a function of instruction or information presentation. Thus we did not consider directly the expanding literature on hypermedia interface design where speed and accuracy of location are the primary dependent variables, although studies employing these variables were included where the emphasis on learning and comprehension of material was paramount. For this review we considered hypermedia to be a generic term covering hypertext, multimedia, and related applications involving the chunking of information into nodes that could be selected dynamically (McKnight et al 1991).

As such our focus was deliberately narrow. Studies emphasizing either the development process underlying hypermedia applications (included in Chen and Rada, 1995 and Landauer, 1995) or those that report qualitative learner/instructor responses were not included in the present review. While there are many accounts of hypermedia development and design (e.g., Kahn et al, 1995) these rarely afford details of subsequent use by learners and thus have little relevance to this discussion. Similarly, it is not our intention to enter the quantitative versus qualitative debate (which is seen by the present authors as uninformative), but to emphasize the quantitative studies of learner performance in order to provide a benchmark. A subsequent review of qualitative studies is both necessary and indeed desirable, but at this time the present authors believe that a clear statement of quantitatively measured effects will offer the most insight, and with the literature increasingly being published across disciplines, a synthesis and distillation of findings to date seems necessary. Furthermore, as we will show, the literature on hypermedia has expanded so fast, probably faster than many of us realise, and thus reviewing completely the more than 2000 articles in this domain is beyond any one paper.

Research Methods
The review concentrated on research findings published between 1990 to 1996 and abstracted or cited in the following periodical indexes; (1) Educational Resources Information Center (ERIC) database, and (2) PsychLIT database. These databases were selected as representative of the core literature indexes in the areas of education and learning. We found few pre-1990 studies on hypermedia that warranted inclusion and wished to avoid criticisms of basing conclusions on 10 year old technologies that may have significantly evolved. Naturally, new findings are appearing all the time, and the established databases serve as a filter on the literature in both negative and positive ways, but our purpose is to show what is being picked up and categorized in the field (itself an important quality control process) and any slight time-lag this involves is indicative of all research domains served by the academic databases.

Multiple searches were performed on each database. Searching on the queries hypermedia and learning and hypertext and learning yielded 397 citations in ERIC alone from 1990-1995. A second ERIC search using the keyword query hypermedia.maj. and (learning or (instructional.adj (effectiveness or design))).maj. resulted in 101 citations which were compared to the first set of results and duplicates were eliminated. A PsycLIT search was initiated using the keyword query (hypertext or hypermedia) and (cognit* or learning or study) which resulted in 63 citations. Each citation was reviewed to determine conformance with the criteria defined earlier in this article. This left a combined list of 97 articles to be reviewed in detail against the selection criteria. After reviewing these, we found 25 that warranted detailed review for this paper.

A final round of searches were completed in Fall 1996. The ERIC database was searched using the keyword query hypermedia.maj. and (learning or (instructional.adj (effectiveness or design))).maj with the dates limited to 1995 through 1996 (to capture new entries) which resulted in twenty-one citations. The PsychLIT database was searched using the keyword query (hypertext or hypermedia) and (cognit* or learning or study) resulting in six. The two lists were combined and duplicates eliminated. The new list provided five new articles for a total of 30 that finally met all criteria in this study.

Supplementary papers cited by authors reviewed and/or known to the present authors through their own research works were included where they offered unique perspectives to this main body of work. While this is not a large document set, it represents a substantial increase in the number on which Landauer (1995) based his conclusions (precise overlap is impossible to assess since Landauer does not provide a full list). The final set employed included only 3 of the 18 articles reviewed by Chen and Rada, the remainder of their set being rejected on the grounds of being pre-1990 (6 articles), not directly measuring learning outcome (6 articles), or being unpublished in the mainstream literature (Chen and Rada include 3 unpublished dissertations in their set).

It is easier to understand the basis of exclusion by offering an example. We did not include a study by Frey and Simonson (1993) who employed a HyperCard document on historical costume to investigate the relationship between cognitive styles and media use in a hypermedia treatment. Eighty undergraduates students were given a Learning Style Profile (LSP) to establish a cognitive style baseline, though the 'styles' they refer to are more appropriately skills (e.g., analytic skill, memory skill etc.). In addition, they were given a test to establish their level of prior knowledge about the subject matter. After the hypermedia lesson, students were given a post test to determine knowledge about the subject matter. The authors found that knowledge of subject matter did increase significantly after use of the hypermedia treatment compared to pre-test scores though it is not clear precisely what knowledge, if any, students had at the start. While certainly quantitative, few conclusions are capable of being drawn from this study since no controls (in terms of learning environment or student ability) were employed. Such studies are typical of published accounts of this technology.

At the outset it should be noted that synthesizing this literature is an exceedingly difficult task. Chen and Rada (1996) grouped all their studies under headings of effectiveness and efficiency regardless of the variables measured, but such an analysis, it is argued here, is impossible to apply meaningfully in this literature. For a start, most of the results in the education literature are not significant. Furthermore, as Chen and Rada noted, most of the experimental studies of hypermedia use unique applications (frequently designed by the experimenters themselves), investigate distinct learner populations (varying in age, skills, experience, ability, learning style and all combinations herein), on distinct learning tasks, with learning outcome or the process of use quantified differently. As a result, it is frequently necessary to describe in detail the investigative methodology of specific studies. However, the general review can be broken into three major themes, each representing an issue of learning that groups of researchers have focused upon directly:

(1) comprehension of presented materials;
(2) learner control over the presentation of material;
(3) individual differences in learning style.

Comprehension is a classic outcome measure of performance and perhaps the strongest test of a learning technology. In these studies, researchers compared hypermedia with other media such as paper, or compared various hypermedia versions of information, and measured the performance of learners with these tools. The second theme is a process issue relating to the control of presentation, pace and movement through the information space etc., variables which are thought to improve the sense of control a learner has over their task and, theoretically, is thought to affect learning outcome positively (e.g., Landow and Delany 1991). The third issue is a form of individual difference analysis, with the focus on types of learners for whom certain forms of hypermedia might offer specific learning advantages.

Certainly this division is overly clear cut as several studies measure components from more than one group. The use of this categorization here is primarily as an aid for the reader. However, the presence of these themes in the literature does reflect an awareness that simple measures of learning are unlikely to provide complete answers to the complex question of how hypermedia affects learning.

There appears to be no clear adoption of one learning theory in this research. Many of the studies seem to reflect pragmatic rather than theoretical concerns. Exceptions appear in the work of Jonnasen and Wang (1993) who attempt to relate hypermedia use to the formation of cognitive structures in a classic schema theoretic manner, or the work of Jacobson and Spiro (1993) who propose perhaps the richest theoretical model in this domain - their epistemic beliefs and preferences model - to examine the role of hypermedia in learning complex, cross-referenced knowledge. Beyond these articulations of theory, there are standard adoptions of cognitive style and individual difference perspectives on learning, but no other formal learning theories are made explicit in this literature. This issue is examined further in the discussion section.

Comprehension
When someone reads a text or participates in a class, it is generally assumed that they end this process with some knowledge or information they previously lacked. Hypermedia presentation is considered to improve comprehension by virtue of is capability to support structured access, rapid manipulation and individual learner control. Comprehension measures thus seek to estimate this gain in knowledge. However, as several researchers have noted (van Dijk and Kintsch, 1983; Dillon, 1992), there is no universally agreed-upon measure of comprehension, thus comparisons across studies are rarely straightforward. The studies reported in this section differed in their measures, as well as in their methods, e.g., some compare/contrast paper texts and hypertexts, others compare various hypermedia structures. However, all studies report the use of hypermedia environments, and rely on experimental methods for their comparisons.

Hypermedia and Paper
The majority of experimental findings to date indicate no significant comprehension difference using hypermedia or paper. This appears to be the case both for complex (e.g., essay writing) and comparatively simple (e.g., immediate recall) task measures. However, the experimental designs employed and the dependent variables observed make simple descriptions of this conclusion difficult. To aid matters, Table 1 provides an overview in terms of tasks and measures employed for all the experiments that are discussed explicitly in this section. As will be noted, the most frequent finding is one of no significant difference between the media, regardless of the investigative methodologies employed.

It is worth considering the first four studies in Table 1 together. In each, learners were allocated to paper or hypermedia environments and, on the basis of their exposure, tested for comprehension afterwards. In all cases, no significant performance differences on comprehension tests were observed. However, despite the general similarities, there are task differences between these studies. Aust et al (1993) compared students using one of four learning environments involving permutations of paper or electronic text and their equivalent mono or bilingual dictionaries. Eighty undergraduate students who were enrolled in a fifth semester Spanish language course were randomly assigned to one environment. Using a 420 word Spanish text judged to be of moderate difficulty by two experts who identified 65 propositions in the text as units of measurement, a paper and pencil post-test asked each student to recall as many of these propositions from the article as they could.

Obviously the text and task in the Aust et al study were narrow and perhaps unrealistic, but in a more elaborate test over several semesters, and employing a lengthier digital document, van den Berg and Watt (1991) developed what they termed a 'Level of Abstraction Structured Text' (LAST) covering introductory statistics and hypothesis testing. LAST documents are hierarchically organized hypermedia in which the frames (pages) form a logical tree, thereby providing a guiding structure for learners.

During the first semester, 28 students were randomly assigned to use the LAST document for six weeks instead of going to lectures, while a control group attended lectures. In the second semester, 30 students were randomly assigned to use the LAST document as a supplement to the lectures while a control group only attended the lectures. During the third semester, the entire class used the LAST document as their sole instructional source. It should be noted that all students in the first and second semesters attended lectures for the first five weeks. The authors remark that on the basis of exam results:

"there was no consistent difference between the students who used the LAST and those who were in standard lectures. In addition, the level of performance did not differ across the three instructional settings. Hence would appear to be no basis for choosing hypertext over traditional lectures, or vice versa, nor for choosing one instructional use of LAST over any other." [123]

Becker and Dwyer (1994) further corroborated these results in their study of undergraduate business students in a beginning course on auditing and computer viruses. The authors developed two treatments, a paper packet and a hypertext program. A pre-test ensured a uniform level of background knowledge among students. Two sessions were scheduled, the first covered the computer virus material and was used to provide those students' using the hypertext programs with some experience using hypertext. The control group simply read the virus material. The second session covered the auditing material. After completion of both sessions a post-test was given to the students. These authors also found no significant difference between the post-test scores of hypertext group and the paper group.

Utilizing the essay writing method of comprehension assessment, McKnight et al (1992) also reported no significant difference between graduate students' performance after exposure to material presented in hypermedia or paper. Using the GUIDE TM application package for hypermedia presentation, these authors required students taking a course in ergonomics to study a 10 000 word document on user-centered systems design. Students were randomly allocated to either condition and told to take as many notes as they required with a view to producing a synopsis of the article after the task. After one hour, all learners were required to stop reviewing the document and to start writing. A domain expert, unaware of the experimental conditions, graded the essays.

While the exact tasks and the dependent variables are different in each of these studies, the effect of the task itself was not systematically manipulated in any of them. Since hypermedia form a powerful means of manipulating large amounts of data, presumably tasks that require such actions are likely to be better supported in the electronic domain than on paper. Tackling precisely this issue, Lehto, Zhu, and Carpenter (1995) compared learner performance on a 'reference' task and a more traditional 'learning task'. After training, 15 graduate students performed two tasks: (1) a 'reading-to-learn' task (which required students to browse and comprehend) and (2) a 'reading-to-do' task (which required students to find and record which annotations contained information on a topic). Two reading-to-learn questions were to be answered using the hypermedia treatment and two using the paper treatment. Time to form topic, time to answer (summarize), and percentage of relevant references cited by participants were recorded. After completing the reading-to-learn task, the participants were asked to complete 10 reading-to-do questions which were randomly divided between the hypermedia and paper treatments. Time to find relevant references and percentage of relevant references cited by participants were then recorded.

The authors reported that in the reading-to-learn tasks, paper users provided significantly more correct references than did hypermedia users (though this result is reported as significant only at the p<.10 level). Hypermedia users also took slightly more time to form a topic but slightly less total time to answer questions, though neither of the last two findings were statistically significant. The authors conservatively note that "taken together, these results clearly do not show an advantage of hypermedia over the book for the reading-to-learn task" [304], though one might add that the significant effect for correct references rather unambiguously suggests a distinct disadvantage for hypermedia users.

Interestingly, the results of the 'reading-to-do' test showed a statistically significant (this time at the standard p<.05 level) advantage for hypermedia on measures of time to complete task and percentage of correct references cited. The authors explain this finding in terms of the more flexible search strategies hypermedia makes possible though it is not clear from their data that strategy differences explain the findings as elegantly as the speed and power advantages of electronic searching, since the 'to-do' tasks seem to have been little more than word searches. Regardless of the reason, the task dependency of these results suggest one clear direction for further research, location of target information in large documents, as opposed to broad comprehension measured after exposure, where hypertext's increased functionality may offer advantages.

It could be argued that the studies reported so far compared the two media for constrained or small information resources and the pattern of results observed may, in part, reflect this. Where much larger information resources are employed, the advantages of hypermedia might become more obvious, if only because of the advantages of searching and speed of access afforded by this technology. Fortunately, results from a study of an extremely large information resource have been published.

The Perseus Project is a Hypercard program consisting of texts (including original full Greek texts of seven authors, partial texts of 3 more, and English translations of all these, historical background material), language tools (Greek-English lexicon, etc.), approximately 30 000 images, and reference tools. Included is a path tool allowing users to follow or create and store paths through the material. This hypermedia application uses, for the most part, implicit links as the designers felt explicit links represented an editorial act which would inhibit the environment they were creating. Perseus was based at six locations and over three years 640 students and 20 instructors participated in the field tests. Using the Perseus Project as their hypermedia treatment, Marchionini and Crane (1994) explored what they terms as the three important characteristics of learning and teaching in a hypermedia environment: access, freedom, and collaboration.

Obviously such a resource is a wonderful test case for hypermedia and there are multiple variables that could be examined. For the purposes of the present review the major points to note are that Marchionini and Crane (1994) report quantitative results from two specific studies: the first compared 10 students using the Perseus lexicon and another 10 using a paper lexicon during a translation task; the second compared 10 students using Perseus to study several Greek plays and a control group which did not use Perseus to cover the same material. The authors reported no significant differences in total time to conduct searches in Perseus or on paper, and more importantly, no significant difference between students on traditional measures of critical thinking (essays and translations). However, the number of citations present in student essays was significantly greater for students using Perseus, an observation in line with the Lehto et al (1995) finding that locating references seems to be enhanced in hypermedia environments.

While some authors have interpreted the lack of difference between the media as a sign of progress (e.g., McKnight et al (1992) considered their result a triumph for the user-centered design process they had followed, indicating that hypermedia could at least produce results equivalent to paper), two published studies have produced significantly positive comprehension results for hypermedia. Psotka, Kerst, and Westerman (1993) devised two experiments covering information contained in the Army Field Manual on recognition of aircraft types. In their first experiment, 10 undergraduate students were either given a paper or hypermedia copy of the manual and 30 minutes to learn 20 airplane types. Paper manual users were given instruction on how to use the manual to make side by side comparisons of aircraft types (pages were folded in half and compared to other folded in half pages). At the end of 30 minutes, a post-test was administered. The hypermedia group out performed the paper group with a mean score of 17 to 12.8. This effect was replicated in a further study where subjects were asked to generalize their knowledge of aircraft types by identifying the same set of aircraft from totally new photographs (a task more akin to comprehension than the original identification/recall task). The researchers attributed their findings to the functionality of rapid access in hypermedia which enabled learners to develop a better sense of similarities and differences between objects.

However, it must be noted that the two versions of the material were not well matched. The hypermedia tool provided color contrasting, supporting the superimposing of one airplane over another for comparison of shape, apparent motion contrasting (one airplane presented after the other for similar planes, in rapid succession) and similarity clustering of like objects in the visual display. While the results suggest that the digital version can certainly be designed to improve on typical paper versions, the scope for improving the latter was not explored. More than anything else, we can conclude that in visual categorization and discrimination learning, the use of animation and superimposition made possible in hypermedia clearly has important impact on learner performance. However, while hypermedia supports such forms of presentation, they are by no means unique to this technology.

In a more standard comprehension task, Blanchard (1990) examined community college students using a hypermedia system to learn MS-DOS. The objective was to have at least 80% of the students pass the proficiency test after eight weeks of class. Students were given a pretest at the beginning of the class to determine their level of knowledge. Unfortunately, the control group, which used the traditional lecture/lab method, did not take the pretest. Blanchard found that the four classes using the hypermedia system had a greater number of students' passing the test and had higher average test scores than the control group. In the four classes using the hypermedia system there was a significant increase in the percentage passing and in the average test scores between the pretest and the posttest. Obviously no pre-test/post test data were available for the controls.

While Blanchard's study is flawed methodologically, the combination of Psotka et al's (1993) finding on visual learning, Lehto et al's (1995) results with respect to reading-to-do and Marchionini and Crane's finding on reference citation, suggests a strong task dependency to the successful exploitation of this technology. So while it seems that paper offers significant advantages over hypermedia in some comprehension tasks, those tasks (or sub-tasks) that involve substantial amounts of large document manipulation, searching through large texts for specific details, and comparison of visual details among objects, are potentially better supported by hypermedia.

Structural Comparisons of Hypertexts
While comparisons of paper and hypermedia suggest there is no simple answer to the question of which medium is better for learning, the ease with which one may organize and structure material has long been considered a potential advantage of hypermedia presentations. Indeed some researchers (e.g., Smith, 1994) have argued that hypermedia can model the knowledge structures of experts in a manner that makes their assimilation by learners more likely. The argument appears to be that hypermedia may support the production of more effective pedagogical resources if they are designed in such a way as to model knowledge structures explicitly. By extension, the relatively poor showing of hypermedia in comparisons with paper may be explained by the failure of those researchers to design applications in a manner which examples learners to exploit such structures. Three studies have examined the precise effects of various hypermedia forms on learning, and their results are summarized in table 2.

<Insert table 2 here>

Jonassen and Wang (1993) argued that hypermedia's support of structural mapping would lend itself ideally to helping novices acquire an expert's representation of a subject domain (a view termed the 'naive associationist' model of hypermedia by Dillon, 1996). These authors tested this theory on pre-service teachers in an education program where their task was to learn about the classroom use of hypermedia as a new instructional technology. These authors developed three test instruments consisting of 10 questions each covering the following aspects associated with structural knowledge: (1) relationship proximity judgments, (2) semantic relationships, and (3) analogies. An additional multiple choice test was developed to assess student recall of the information contained in the hypermedia. These test instruments were then utilized in three studies examining the effects of various hypermedia structures on learning.

In the first experiment: a graphical browser, a pop-up window (both of which provided explicit information about the structure of the document and the nature of the links), and a control hypermedia (where no extra information was provided)were compared. Of the four dependent variables measured (recall, relationship, proximity, and analogy subscales) only the recall variable approached statistical significance (p<.07), and this was in favor of the control group.

In the second experiment, Jonassen and Wang tested the value of generating a classification of the link by the students. One hundred and twelve students were divided into three groups (generative, pop-up, and control). In this experiment, the link relationships were not displayed to the generative or pop-up groups, instead students in these groups were required to classify the link themselves. The rationale here was that forcing learners to attend to the link type would enhance their perception of the semantic structure of the document. If after two tries the students were unable to classify the link, the program provided the link type and proceeded to a new node. The authors observed that "as in experiment 1, control group subjects were better able to recall information, as they were less distracted by the structural knowledge activities. Neither structural strategy (generative or pop-up) produced any increase in structural knowledge." [p5]

Jonassen and Wang's third experiment involved forty-eight graduate students divided into two groups. The authors argued that actively engaging the learner in their own learning activity should enhance performance. As a result, group one was required to construct a semantic network of ideas from hypermedia material. They were provided a tool with which they already had experience to accomplish this task. A control group was given the task of simply studying the material in the hypermedia. The treatment group performed significantly better only on the relationships judgment task than the other group, suggesting (perhaps) that learners need to become focused on structural relationships to acquire structural knowledge. The authors note that "merely browsing through a knowledge base does not engender deep enough processing to result in meaningful learning" [p6], and conclude that their initial assumption of the assimilation of expert knowledge being easier in a hypermedia environment is not supported. Perhaps most insightful is the authors' contention that hypermedia might work better as an information retrieval interface rather than a learning enhancing tool, in line with the findings of Lehto et al (1995).

While Jonassen and Wang's results cast some doubts on the simple view of knowledge transfer being enhanced through structural manipulations in hypermedia presentations, it could be argued that the type of knowledge that one seeks to learn is a crucial variable, and that the three Jonnasen and Wang studies might not have utilized the most appropriate knowledge type. Jacobson and Spiro (1993) explored such an issue by examining students using hypermedia to learn both complex and ill-structured knowledge. Three environments were developed: two control conditions which presented the same material in a more rigid manner (minimal hypermedia/computer-based drill) than the experimental treatment: a full hypermedia. It was hypothesized that the control groups would achieve higher scores on the memory tests of factual knowledge while the full hypermedia users would provide higher scores on the transfer test. The authors also hypothesized non-specific user preference differences.

Thirty-nine freshman and sophomore students were given a pre-test to determine prior knowledge and then randomly assigned to one of the three conditions. The material covered was characterized as being conceptually ill-structured due to its multidisciplinary, diverse, and dynamic content. Learners engaged the technology for 4 sessions, one per day, with testing of their performance taken on days 2 and 4. Testing involved recall of factual material and a written essay. The control groups were significantly more effective and efficient in acquiring factual knowledge as evidenced by their scores on both recall tests. However, the experimental group out performed the control groups in the problem solving essay on day four, though no significant difference was observed between groups on their day-two essays. The authors suggest this pattern indicates the task dependency of hypermedia use - it seemingly is of little use, and may even be disadvantageous for abstraction of factual knowledge (though is not clear why this should be so theoretically), having more useful application for learning tasks involving synthesis of complex material.

Both the aforementioned investigations allude to the dynamics of comprehension in hypermedia environments. The tasks were reasonably complex but despite numerous samplings of learning measures, few significant differences emerge. What sets these two studies apart is their explicit articulation of theoretical issues, albeit two very different theories. Jonassen and Wang clearly adopt a structuralist perspective which is seductive but limits our analysis of comprehension to one based on form more than content. Jacobson and Spiro's position is more subtle and allows for a more complex model of learning that hypermedia may support. If progress is to be made in this area it is likely that the latter authors' approach is the most compelling (an issue we shall return to later).

Using a comparatively simplified learning task, Tripp and Roby (1990) concentrated on the effects of hypermedia on acquisition of Japanese words. They developed four hypermedia test environments that were intended to manipulate the cognitive processing of learners by crossing the use or absence of an advance organizer for the material with the presence or absence of a visual metaphor about the organization of the database. However, the details of these treatments are somewhat vague, and the visual metaphor seems to have been a background graphic on the screen. They hypothesized that students would learn fewest Japanese words in an unstructured hypermedia environment that lacked organizer or metaphor since these devices should free up limited cognitive resources for learning.

Sixty undergraduate and graduate students were randomly assigned to one of these environments with the task to learn as many Japanese words as possible in 15 minutes (a task that appears to be lacking in ecological validity). A paper test consisting of multiple choice and recall questions was given at the end of the session. Learners who were exposed to both the metaphor and the advance organizer actually did significantly worse then either group which had only one of these treatments and only slightly better than group with neither of these treatments, confounding the authors' views of the likely cognitive benefits of such treatments. Tripp and Roby concluded that "two types of orienting devices activated conflicting mental models of the lexicon," [122] therefore, while orienting aids such as advance organizers or metaphors may be of some benefit in the learning task, more is definitely not better.


Hypermedia and other electronic media
As well as considering the best form of hypermedia, there has been interest in comparing hypermedia with other electronic media. Saga (1992) explored the effects of hypermedia and video environments on learners' interest and comprehension. Adapting a hypercard-videodisc program from a video presentation entitled Great Authors Who Lived in Bunkyo, three subject groups were recruited: 57 college students, 20 audiovisual librarians, and a control group of 17 college students. All subjects were given a pre-test measuring their prior knowledge about the subject. The two treatment groups participated in an hour-long presentation which included a lecture, the hypercard-videodisk presentation, and the video. The control group only watched the video. All three groups showed a significant increase in knowledge in the post-test results. However, the control group out performed the other two groups in a test of factual knowledge. In a conclusion that is fitting for this entire section of the review, Sage notes that this finding is "impressive enough to warn against excessive expectations from hypermedia." [186]

Summary of comprehension findings
The results on learner comprehension from hypermedia are at best inconclusive, but the weight of evidence points to hypermedia being suitable mainly for a limited range of tasks involving substantial searching, or manipulation and comparison of visual detail where overlaying of images is important. In short, the evidence does not support the use of most hypermedia applications where the goal is to increase learner comprehension (however measured). Evidence from studies of hypermedia structural variables suggest that knowledge of how best to organize information in digital form that exploits the cognitive capabilities of learners to link and organize new information is particularly limited.


Learner Control

If hypermedia use does not lead directly to gains in comprehension, it might be argued that the medium's advantages really lie elsewhere i.e., hypermedia's effects are mediated by other variables. For example, the capability of digital technology to enhance learner control over the pace and detail of information delivery has been claimed to have a positive affect on learning by many authors (Dryden, 1994; Landow, 1992; Landow and Delany, 1991, though see Hooper and Hannafin, 1988, and Relan, 1991 for alternative viewpoints). Since one of the advantages of hypermedia is precisely the control it allows users to have over their access to information, changes in learning outcome might be best observed under conditions where learner control is affected. Five published studies have examined this issue, and we examine these in turn.

Control of access and learner performance
Manipulating the learner control variable is not a straightforward matter. Most researchers attempt to do this by creating various presentation formats that vary the means by which the learner may manipulate the information displayed. In this way, a hypermedia application with multiple links and/or a graphical browser that offers selectable links is seen as offering more control to the user for access to material than a simple text file that only affords screen scrolling or page turning. While objections may be raised in terms of the validity of such operationalisations of learner control, and it is certainly the case that no fixed scale of controllability is being used here, it is likely that such treatments offer at least one plausible, if partial, manipulation of learner control.

A typical example of such an investigation is reported by McGrath (1992) who investigated the role learner control played in reducing misconceptions during learning. One hundred and three undergraduates in a teacher preparation course were randomly assigned to one of four test environments: hypermedia, paper, menu-driven electronic pages, and no menu electronic pages (where subjects had no choice but to go on to the next page). A pre-test to determine a prior knowledge was administered which classified each student as either high or low ability and highlighted any previously held misconceptions held by the student on the subject matter. A post-test was administered to determine student achievement.

McGrath hypothesized that the increased control of the hypermedia and menu-driven pages would positively affect learning. However, no significant differences were observed between conditions. Comparisons between high and low skill learners across all variables were not statistically significant either. Learners made nonsequential responses in both computer environments which may be interpreted as a sign of manifest control, however, both the high and low skill learners made more such responses in the menu environment then in the hypermedia environment. While these cast doubts on the value of control on certain learners, there is another possible interpretation i.e., the increase in user control that is assumed to occur in hypermedia environments might be false.

In a further examination of learner control in electronic environments Welsh, Murphy, Duffy, and Goodrum (1993) devised three experimental conditions: (1) No Link Type (NLT) where all link icons are the same and do not provide any cues as the information behind the link; (2) Link Type (LT), where different link types indicate information type; and (3) Submenu (SM) where a link leads to a submenu with elaborations listed. The authors predicted that NLT users would manifest random exploration of the database compared to LT and SM users who would target their exploration, following or rejecting links because that would have some idea of the information to be found at the link. They also predicted that NLT users would experience frustration and would follow fewer links than LT or SM users and (unrelated) that high link density on a given page would increase visual noise and disrupt the learner's ability to read form the screen. A 2,000 word core hypermedia database with 320 elaborations was created and 108 undergraduates were randomly assigned to one of the following six environments: NLT-high density, NLT-low density, LT-high density, LT-low density, SM-high density, and SM-low density.

Learners were tested in groups of 20 for about 30 minutes. Comparison of link type and time on task failed to demonstrate any statistical significance, nor did the issue of link density. The authors reported that the NLT condition did stimulate random exploration (though it is not clear how they measured the random variable). Contrary to predictions, NLT users did not experience frustration or become discouraged from the lack of information. The authors concluded rather generally that "designers of hypermedia learning environments will have to make tradeoffs in creating systems that promote exploration and in creating systems that have high usability." [33]

Lanza and Roselli (1991) manipulated learner control in their study of student achievement in either computer-aided instruction (considered low- learner control) or hypermedia (high learner-control) environments in programming language learning. Sixty undergraduate students, with no experience in either computer-aided or hypermedia instruction, were randomly assigned to one of the learning environments. Unlike most studies of hypermedia use, the learners in this study were observed over an extended period. Covering the same material, the students spent four hours a day for two weeks in each learning environment (a trial period was allotted in order that the students would become familiar with the learning environment). A ten question test was given at the end of the two week period to measure achievement. The authors reported a greater spread (variance) in the scores of the hypermedia users, leading them to suggest that hypermedia may not be for every learner.

Quade (1993) also examined the role of learner control in a comparison of a Computer Assisted Instruction (CAI) tutorial allowing only linear movement through the tutorial, and a hypermedia tutorial with a graphical map. Hypermedia users had the option of moving linearly through the material or use the graphic map to select a topic out of sequence or return to the main menu. Both tutorials provided an immediate feedback loop for learners.

Seventy-six undergraduates were randomly assigned to one of the two treatments and none had any prior classroom work in the subject area. Each student's ability was ranked by their professor based on observation and assignment and test performance over the previous eight week period. A 30 question pre-test was administered and students not familiar with the Macintosh platform were given a two hour training session. At the end of the experiment, the pre-test was re-administered as a post-test. Quade reported that learner control over the number of screens viewed was not found to be a factor in the overall subject performance. Furthermore, no significant difference in overall performance (as measured by the post test) was observed.

If learner control on its own is insufficient or difficult to manipulate, its combination with other variables might offer some clues as to the relative merits of this technology. In a study of learner control and the advice available to learners in a hypermedia environment, Shin et al (1994) manipulated the amount of learner control (free access to all contents or limited access through links), the presence of 'advisement', and the level of prior knowledge. In this experiment, advisement gave users recommendations on which sequence to follow in the hypermedia and visual aids for locating themselves in the hypermedia environment. Manipulating advisement and access combinations in a 2 x 2 design, one hundred and ten second graders were assigned to work on a lesson concerning food groups were subjects for this study. A pre-test assessing prior knowledge was given each student. Students were randomly assigned to one of the four combinations. After completion of the lesson a post test was administered. The same post test was repeated one week later.

Interestingly, the authors reported that students in the limited-access treatments answered more questions correctly than did the free-access students (the authors note that these results differ form the Lanza and Roselli findings reviewed earlier). Access had no significant effect on students with a high level of prior knowledge but low level prior knowledge students were more successful in the limited than free-access environments, suggesting an interesting interaction.

Advisement had no significant affect on learning. It did, however, affect time-in-lesson as students with low levels of prior knowledge finished the lesson quicker when they were in one of the no advisement environments (this result seems to have occurred because they quit programs without knowing that there was more information to cover). The free-access/no advisement environment was difficult for all students but especially those with low prior knowledge. Shin et al recommended that limited access should be employed to improve young students' achievement particularly if they have limited prior knowledge. In a conclusion that seems typical of all educational theories, Shin et al state "learners with different levels of prior knowledge require different kinds of instructional approaches." [45]

Conclusions on Learner Control and Hypermedia
With its embodiment of structure and linked information nodes, hypermedia applications are considered to offer users far more control over access and exploration. Obviously control can be manipulated in multiple ways, and the degree of control any one application embodies is difficult to measure, with most researchers manipulating this variable through the provision of selectable links and paths. Different students seem to react to this increased control differently, with lower ability students manifesting greatest difficulty in exploiting it to their advantage. As a general characteristic of hypermedia environments, the ability to control pace and delivery of information, even when coupled with selection advice, appears insufficient to effect learning outcome significantly for all but high ability learners.

Individual differences among Learners.

While the search for learning gains based on media charactersitics or interface features has been largely unsuccessful, a third line of research has focused more directly on characteristics of the individual learner. There have been hints of the importance of learner variables such as level of prior knowledge, for example, in the work of Shin et al (1994), and experience with hypermedia has been controlled in some studies (e.g., Lanza and Roselli, 1991). Extending this consideration of learner variables logically, a variety of individual differences among learners has been proposed as crucial mediating variables in explaining the effect of hypermedia on learning outcome by several authors.

Typically, ability is considered a major determinant but in this literature, learning style (conceptualized as reflecting one's distinct approach to learning) is frequently postulated as an important variable. Learning styles reflect a learner's position on a continuum running between extreme traits such as holistic or analytic, verbal or spatial, reflective or impulsive, or exploratory or passive. Each trait is seen as having advantages in certain situations, but individuals are supposedly characterized as being predominantly more one than the other.

It is not difficult to envisage how such variables might be important and why they are tempting to employ in this domain. The flexibility of hypermedia technology renders it a strong candidate for tailoring the presentation of information to suit particular learners, thus any empirically demonstrated relationship between such an individual difference variable and effects of hypermedia instruction might offer an explanation for the mass of non-significant results to date.

In the present section 10 published studies on the learning style differences in using hypermedia are reviewed (see Table 3).

<insert table 3 here>


As shown in table 3, individual differences have been studied on both large and small samples of learners, and significant interaction effects have frequently been observed. However, not all individual difference variables have proved insightful, and in this section, a broad distinction is made between general ability and specific cognitive style differences as they pertain to hypermedia use.

Learner level and hypermedia use
Obviously the greatest source of individual difference between learners is their general intelligence or level of ability. According to Dillon and Watson (1996), meta-analyses of individual differences studies indicate that general ability is the single best predictor of performance on most tasks, and these findings have relevance to all forms of human-computer interaction also. In the educational literature, general ability has been used as an independent variable in several studies of hypermedia use.

Focusing on the strategies learners use when studying instructional materials prior to problem solving, Recker and Pirolli (1995) developed two instructional environments (hypermedia and a non-hypermedia electronic text) covering the same material (Lisp programming). These authors were interested in the effect on learning of adding elaborations to the instructional material. However, noting that previous studies of the elaboration variable had failed show advantages, Recker and Pirolli further manipulated this variable by providing the learner with the option of viewing or not viewing elaborations.

Sixteen students with minimal programming skills were recruited for this experiment. During the introductory phase, students worked through four lessons followed by a problem solving session on a computerized Lisp tutor. The performance of each student on the last set of problem solving exercises was used to determine an ability or 'level of knowledge' measure. In the target phase of the test, students were randomly assigned to either the hypermedia or control environments. They progressed through four more lessons before another problem solving session on the Lisp tutor. The number of errors occurring during the problem solving session was employed as a measure of student performance.

The authors reported no significant difference between environments. Using their ability measure to distinguish students indicated that the hypermedia environment was mostly beneficial to high ability learners, and that low ability learners' performance decreased after using the tutor. Recker and Pirolli argued that the lower performing subjects were not able to take advantage of the hypermedia and, in fact, may have been overwhelmed by the amount of learner control it provided (echoing an earlier finding of Lee and Lehman (1993)).

Specifically examining lower ability students, Higgins and Boone (1990) explored the effects of hypermedia study guides on students diagnosed with learning disabilities (LD), remedial students, and 'regular' students. The study guides were designed to cover material on the history of Washington State in 10 chapters with links throughout each chapter to more detailed information and a set of study questions followed at the end of each section. The study guides were designed to provide increased instruction time for students without increasing the teacher's instructional load.

Forty ninth grade students, of whom 10 were students with learning disabilities (LD), 15 were remedial students, and 15 were regular students, were randomly allocated into three conditions: lecture, lecture with study guide, and computer study guide only. A pre-test was given to all students to establish their base level of knowledge about the subject. The material covered was the same in all three groups. Those students in the lecture-only group had worksheets with the same information and questions as contained in the hypermedia study guides. Daily quizzes were administered and a post-test was given at the end of the course. In addition, two weeks after the end of the course, the post-test was given again to measure retention.

Unfortunately, the authors report many differences in daily quiz scores without alluding to their statistical significance or indicating how many such differences might be found by chance alone, though in general, regular students significantly outperformed LD in posttest and retention tasks as expected. For reasons that are unclear, even though no significant treatment effects were observed the authors' state: "there is evidence that the computer study guide treatment was as effective as the lecture condition" (p. 535.). Oddly, the authors report that in their pre-test, significant differences between all student groups were observed in the expected direction (regular students outperformed remedial students who also outperformed the learning disabled), except for the computer study guide group. This suggests that the least learning disabled of the LD group were allocated by chance to the computer condition, suggesting even further caution in interpreting these data.

Higgins and Boone then performed a second study using the same computer study guide to see if it could bring 5 failing students closer to a passing grade. Taking these students performance in study 1 as a baseline, they exposed these students to 10 further computer study lessons, and measured their performance again. Results showed that the students raised their grades on the daily quizzes, post-test, and retention test (mean 41% to 52%). Without resorting to inferential tests, the authors concluded that the hypermedia treatment was useful in raising the grades of these lower achieving students although without control measures, these findings only indicate that improvements are gained by making failing students read and review instructional material again.

Horton, Boone, and Lovitt (1990), using the materials and test instruments described above, studied four learning disabled students. In this experiment, control questions were included (questions which were found on all three tests (pre-, post-, and retention) but where not covered in the study guide). The authors found that as a group there was a significant increase in correct answers between the pre-test and the post-test, and between the pre-test and the retention test, and no significant difference in scores on the computer items between the post-test and the retention test, suggesting a clear immediate learning effect and good retention. As expected, no effect for control items across pre-, post-, and retention tests was found. The authors note that this group of "LD students demonstrated significant improvement on computer questions, but not on control items, ....and those effects were maintained over a four-week retention period." [129] It is not clear how best to interpret such findings, as the necessary control of non-hypermedia study guides was not utilized for comparison. At best it shows, once again, that studying material can improve performance.

Repman, Willer, and Lan (1993) investigated whether novice and low-ability students would benefit from the combination of HBI (Hypermedia-Based Instruction) in a collaborative learning environment. In particular, they wanted to know if working alone was significantly different from working in pairs, and, in the case of paired working, if the composition of the pair (in terms of ability levels) was important.

These authors developed a hypermedia-based instructional unit. One hundred and eighteen students enrolled in a computer literacy course were randomly assigned to work individually or in pairs during a 50 minute class. A paper and pencil test covering the material was administered the next day. The authors reported no significant effect for working alone or in a pair. However, magnet students, working alone or with a peer, out performed all others. Non-magnet students did significantly better when paired with a magnet student than working alone. Perhaps most worrying, the magnet students working with non-magnet students scored approximately one standard deviation lower than working alone or with another magnet student. Repman, Willer, and Lan concluded that students do approach HBI learning "differently" and these differences "vary greatly across social context as well as within each group." [294], suggesting specific further avenues for research.

While it seems that the ability differences among learners certainly account for some of the variance in performance with hypermedia, there are other individual differences that have been proposed as explanatory. Learning style dimensions are considered independent of ability (though correlations between some of these and ability have been reported (see Dillon and Watson, 1996) and they remain popular as they may offer the clearest indication to educators of how hypermedia interventions can best be targeted at specific learner populations.

Learning style: Field Dependence/Field Independence
A popular source of individual differences seems to be the cognitive style construct of Field Dependence (FD) and Field Independence (FI) which is generally considered to represent differences in preference to attend to specific issues or to rely on context. Lin and Davidson (1994) hypothesized that structured hypermedia would provide an organizational aid to learning that is likely to be differentially useful to FD and FI learners. These authors identified five types of linking structures commonly used in hypermedia (linear, hierarchical, hierarchical-associative, associative, and random) and investigated whether performance was significantly predicted by linking structure, field dependency/independency (FDI), and their interaction. They then developed five experimental treatments, each using one of the five linking types, covering material on the Tianamen Square Incident. One hundred and thirty nine undergraduates were given a Group Embedded Figures Test (GEFT) to determine cognitive style, and then randomly assigned to one of the five treatments groups. Consistent with other findings, FI learners out performed FD learners regardless of environment. The authors observed no difference in performance between learners with the same cognitive style as a function of linking structure and reported that "performance of these subjects cannot be predicted by the interaction of linking structure types and cognitive style" [459].

This style dimension was also investigated by Liu and Reed's (1994) who asked two questions; (1) What is the relationship of learning style and learning strategy? and (2) What types of media, tools, and learning aids are preferred by learning style groups? The authors developed a hypermedia (incorporating text, sound, video, and hypermedia links) treatment that emphasized meaningful use of vocabulary in the proper context. The treatment consisted of four subprograms with each subprogram containing 20 highlighted vocabulary words. When the highlighted word was selected the learner was provided with the following options: (1) definition, (2) part of speech, (3) sentence examples, (4) video context, and (5) relationship of the word to other words. Also included was the capability to take notes, on-line help, index tools, and a map (navigation) tools.

Sixty three international students in an intensive English program were given the Group Embedded Figures Test to assess their cognitive style (FD, FI, or FM (Field Mixed). At the completion of the session, each student took an achievement test. The authors reported that in the area of learning patterns, FD learners used the various features of the courseware significantly more than FI or FM learners but no significant results on outcome were observed for the relationship between learning style group and media access, or use of learning aids, dictionary, or background information.

Jonassen and Wang (1993) also investigated learner styles as part of their study described earlier and reported a statistically significant relationship between FI learners and the recall and relationship variables measured in their post test, supporting recent suggestions that the FI construct more truly measures intelligence than any meaningful cognitive style (Eysenck, 1990). According to the authors FI learners "were the only learners able to successfully use structural cues to acquire more structural knowledge information…. It is likely that field independent learners are better hypermedia processors, especially as the form of the hypermedia becomes more inferential and less overtly structured." [7]

In a study that encompasses all three of the areas covered in this paper: comprehension, learner control, and learning style, Stanton and Baber (1992) explored the hypothesis that giving learners greater control over the own learning should increase the effectiveness of computer based training. Twenty subjects interacted with a training system on process control that consisted of 8 modules. Learners could repeat modules or choose not to interact with a module if they did not feel they required that piece of training. After completion of the training, subjects' moved on to the "Process Control" plant task. The decisions made by the subjects as well as the status of the plant during simulation was recorded, enabling plant "output" to be recorded as index of task performance. One week later they again interacted with the task (without further training) yielding a measure of the degree of training retention. In addition, the amount of time spent in training, time spent on each instruction module, and time spent in practice modules were recorded.

Prior to testing the subjects were administered a Embedded Figures Test (EFT) to determine cognitive style. A post experiment questionnaire was used to determine their approach or strategy to the training environment. After the experiment, the authors divided the subjects into field dependent/independent groups and learning strategy (top down, bottom up, sequential, and elaborative) groups, based on their node visiting sequences. These authors report significant differences between the cognitive style groups in terms of number of modules visited and completed, and time spent in the training sessions only, there was no significant difference in transfer task performance. The authors note that the relationship between field dependence/independence and learning strategy was not conclusive and admit that the strategies they describe might be just artifacts of their method rather than representations of true user differences, further demonstrating the need for greater methodological rigor in this domain, and confirming the general lack of insightful outcomes of research based on the field independence/dependence style construct.

Learning style: passive/active learners
A learning style dimension of possible direct relevance to hypermedia use is the passivity/activity of the learner. Since hypermedia may support greater direction by the instructor of material to follow, there are grounds for believing that learner passivity/activity may interact significantly with successful learning from this technology.

Lee and Lehman (1993) investigated this construct in terms of control of information delivery in hypertexts covering the topic of DNA and protein synthesis. While both presentations had explicit and implicit buttons linking information, only one had cueing. In the cueing presentation, students were alerted that more information was available if they chose to view it. One hundred and sixty seven undergraduates were evaluated to determine that a priori level of subject knowledge (a score of above 80% disqualified one learner from this test). The remaining students were given a Passive Active Learning Scale (PALS) evaluation to classify their learning style which classifies learners as active if they exhibit curiosity, initiative, and a wide focus while selecting information on their own; or as passive if they select only information overtly provided and demonstrate indifference, dependence, and a narrow focus. Those who fall between the two other types are termed neutral learners.

The authors report that active learners out-performed the others on all three dependent measures and passive learners were out-performed by neutral learners, suggesting some validity to the active/passive construct as a predictor of performance. Cues had no significant effect on active learners, but passive learners showed a significant increase in the achievement variable in the explicit cue condition, suggesting that passive learners may react to prompts for them to follow links. Lee and Lehman advise designers take such individual differences into account.

Shute (1993) developed two environments: an application environment which provided immediate feedback (in the form of the rule) for correct or incorrect responses, and an inductive environment which required learners to discover the rule that determined the correctness or incorrectness of their answer. Three hundred and nine subjects were randomly assigned to one of the environments and two pre-tests assessed prior subject knowledge. At the end of the trial, a series of four post tests were administered to measure acquired knowledge. Shute hypothesized that learners who manifest exploratory behaviors (i.e., were active) in their performance data would demonstrate higher scores on the post test if they learned in the environment which allowed exploratory behavior (inductive) while less exploratory (passive) learners would benefit from a more structured environment.

Shute reported that in the area of learning outcome (percentage correct on the four post tests) neither environment showed a significant advantage over the other. However high procedural exploratory behaviors were significantly associated with poor outcomes, and the proportion of time allocated to reading definitions was a significant positive predictor of learning outcome. Finally, there was a significant interaction involving procedural (but not declarative) exploratory behavior and learning environment predicting learning outcome. Shute notes that when learner style is correctly matched to learning environment learner performance may increase, though a systematic manipulation of this variable a priori would lend stronger support to the exploratory behavior dynamic.

Learning Style: deep/shallow processors
Deep and shallow processing refers to the degree of structural or surface analysis and metacognition learners typically manifest in response to new information. Learners may be divided into deep processors who have learning strategies that relate and structure information actively and surface processors whose learning strategies are more passive, usually centered around memorization and rehearsal of information.

Beishuizen, Stoutjesdijk, and van Putten (1994), developed two treatments to test the hypotheses that students with these learning styles profit differently from different types of instructional materials and that increasing task constraints will affect students with different learning styles unequally. In experiment one, 48 students were given an Inventory of Learning Styles (ILS) evaluation to determine whether they were deep or surface processors before completing a task involving preparation for an examination. They were randomly assigned to one of three conditions where they received: hints about study strategies, content structuring help, or neither of these. At the end of the study period, students were administered a post test to determine knowledge acquisition. No significant main effects were observed for processing style or hypermedia treatment in either process or outcome measures but the authors identified interactions that suggested content structuring help was detrimental to deep processors but helped surface processors. The authors noted that hypermedia materials are suited for those students who "tend to primarily rely on structures they create themselves." [164]

In the second experiment students were given a task that emphasized selection of information and the authors further distinguished learners in terms of 'self-regulation' on the basis of a sub scale in the ILS. The authors hypothesized that surface processors would make more use of the available text-related guiding tools in their hypermedia assignment, while deep processors were expected to use the map function to select text units in the hypermedia. Forty-two students who had been given an ILS evaluation took part in this experiment using the materials from the first experiment. The authors report that students who combined self-regulation with deep processing and students who combined external regulation with surface processing performed better than did students with complimentary combinations of regulation style and processing style. The results of both experiments were interpreted as confirming that surface processors were less comfortable in the hypermedia reading environment.

Conclusion on Learner Style
The interaction of learner style on use of hypermedia offers perhaps the beginning of an explanation for the generally conflicting results in the literature comparing hypermedia and non-hypermedia learning environments. However, the concept of learner style has many meanings and it is not always clear how, e.g., deep processing, self regulators, are similar to high procedural exploratory learners. The cognitive style distinction of Field independence/Field dependence remains popular but, as in most applications to new technology design, it has failed to demonstrate much by way of predictive or explanatory power and perhaps should be replaced with style dimensions that show greater potential for predicting behavior and performance.

What seems to be conclusive is the fact that high ability learners will perform better than low ability learners, regardless of the medium of instruction but hypermedia applications can offer techniques such as explicit cueing that can aid the less able student to perform better. Obviously, this area needs much more research to yield the form of evidence that can drive design or exploitation of the technology but it does suggest that the sue of hypermedia in education should be based on appropriately designed technology aimed at specific learners if any significant benefits are to be obtained.


General Conclusion

So what are we to conclude from the studies reviewed in this paper? Clearly, the benefits gained from use of hypermedia technology in learning scenarios appear to be very limited and not in keeping with the generally euphoric reaction to this technology in the professional arena. To the present authors, the experimental evidence to date suggests three broad conclusions:

(1) Hypermedia affords most advantage for users in specific tasks which require rapid searching through lengthy or multiple information resources and where data manipulation and comparison is necessary. Outside of this context, existing media are better than or equal to the new technology;

(2) Increased learner control over access is differentially useful to learners according to their abilities. Lower ability students have greatest difficulty with hypermedia.

(3) The interaction of learner style on use of various hypermedia features offers perhaps the basis of an explanation for the generally confusing results in the literature comparing hypermedia and non-hypermedia learning environments. Specifically, passive learners may be more influenced by cueing of relevant information, and the combination of learner ability and willingness to explore may determine how well learners can exploit this technology.

From these broad conclusions it can be inferred that the value of hypermedia in pedagogy is limited. As hypermedia is ultimately a form of information presentation, there should be no real surprise here. That manipulating form of delivery produces mixed results is a reflection of the gaps in our knowledge of how best to design media and since most educators are fully aware of the multiple forces that shape learning outcome, we should not pin undue hope on any technology of presentation yielding major breakthroughs on education outcome.

However, in tasks that involve multiple, rapid manipulations of complex material, in multiple forms, where term searching is important, or the ability to overlay images or run animated simulations are involved, then the technology is likely to offer many benefits, all else being equal, if the specific form is designed to be usable. Obviously, combining the technology with innovative classroom use, discretionary collaboration, and self-paced learning may offer further advantage, but as yet these scenarios remain largely unstudied.

Taking the literature as a whole, it is disappointing to report that statistical analyses and research methods are frequently flawed, limiting our understanding of these important issues. Failure to control important variables for comparative purposes, lack of adequate pre-testing of learners, the use of multiple t-tests for post-hoc data and even the tendency to claim support for hypotheses when the data fail to show statistically significant results, all suggest that the basis for drawing conclusions from this literature is far from sturdy.

Coupled with this is a tendency for experimental investigations to limit themselves in terms of tasks, information types, and learners to very narrow areas of learning. Studies that use as subjects individuals who are themselves learning to become instructors (McGrath 1992) or employ as test vehicles hypertexts about hypermedia (Jonassen and Wang, 1993) are self-referential and run the risk of failing to control for innumerable contaminating variables. While the present authors have no wish to prevent studies of instruction being carried out on instructors, for example, it is our hope that such work would be the exception not the norm for any conclusions we draw on learners and hypermedia.

How can this situation be improved? An immediate approach is to concentrate research efforts on those variables that seem most influential. Understanding the various components of learning tasks and identifying those that are likely to benefit directly from hypermedia interventions would be an obvious, human-factors style focus that would surely reap benefits for our understanding (see Dillon 1996 for details). The concept of 'learning' is too broad to be equated with a simple statement about the pros and cons of any instructional medium, and we need to refine our analyses of tasks so that we can ensure greater generalizability of findings on constituent tasks such as information location, concept identification, reasoning, categorization etc.

Thus, as indicated by the Psotka et al (1993) study, careful task analysis of the learning requirement can highlight scenarios where well designed technology can facilitate the learning process. In this fashion, rather than claim a simple advantage or disadvantage for hypermedia in educational settings, efforts could be focused on those components of learning that are amenable to technological support so that maximum advantage of hypermedia's capabilities can be made. This is the manner in which one of the more successful exploitations of this technology was developed. Landauer et al (1993) showed a gain for performance in an open-book statistics examination for users of their hypermedia application (SuperBook) after they have iteratively designed and tested the system to remove usability problems.

A second focus is surely in the area of individual differences among learners. Though Chen and Rada (1996) concluded that differences in cognition alone do not have significant effects, they primarily examined active-passive cognitive style studies, and actually indicate that spatial ability is a medium-large effect. Since hypermedia may be implemented in multiple forms those that are best suited to one style of learner may be completely inappropriate for others. The data reviewed here suggest that this is indeed the case, and a concerted research effort in this area is likely to offer considerable benefits.

Combining task analysis and learning differences in a strong program of empirical research would probably be the best approach. We remain convinced that well designed hypermedia offers the potential to enhance learning in a variety of ways. However, the design variables that are important are not well understood, and they interact with individual differences in learners that at the moment are not well understood. Identifying the precise combination of designs attributes, task domains, and learner characteristics should now be the focus of future research.

Beyond this pragmatic recommendation, there remains a tremendous need for a richer understanding of the learning process beyond how information presentation and access can enhance the educational experience. The use of this technology as a means of information creation by learners might be usefully explored and there is definitely a need to consider the potential for learning with hypermedia, not just from it. Jacobson (1994) makes the case that cognitive flexibility theory (CFT) offers potential insights into how we may best employ this technology and the present authors do not disagree with his theory-to-design approach, though we would make a call for stronger quantitative analysis of this approach. While the present review has the feel of a media comparison review, our aim is more at informing the theoretical development in this field, as it is only by adequately accounting for findings of the kind reviewed here that a theory can claim superiority over its competitors.

Acknowledgments
The authors would like to acknowledge the insightful comments of the anonymous reviewers and Dr. Carl Grant who all suggested changes, raised caveats, and caused us to focus the emphasis of this review considerably.



Bibliography

Aust, R, Kelley, M. and Roby, W. (1993). The Use of Hyper-reference and Conventional Dictionaries. Educational Technology Research and Development. 41(4) 63-73

Barrett, E. (1988) Text, Context and Hypertext. Cambridge: MIT Press.

Becker, D. and Dwyer, M. 1994. Using Hypermedia to Provide Learner Control. Journal of Educational Multimedia and Hypermedia. 3(2)155-172

Beishuizen, J., Stoutjesdijk, E. and van Putten. K. (1994). Studying Textbooks: Effects of Learning Styles, Study Task, and Instruction. Learning and Instruction. 4 151-174

Blanchard, D. (1990). A Hypertext Computer-Assisted Instruction (CAI) Program: Teaching the Disk Operating System (DOS) to Community College Students. MA Thesis Center for the Advancement of Educaiton, Nova University. Available through ERIC.

Chen, C. and Rada, R. (1996). Interacting with Hypertext: A Meta-Analysis of Experimental Studies. Human-Computer Interaction. 11, 2, 125-156

Collier, G. (1987) Thoth-II: hypertext with explicit semantics. Proc. of Hypertext'87, University of North Carolina, Chapel Hill, 269-289.

Delany, P. and Gilbert , (1992) Hypercard stacks for Fielding's Joseph Andrews: Issues of Design and Content. In: Delaney and Landow, G. (Eds.)Hypertext and Literary Studies. Cambridge MA : MIT Press. 287-298.

Dillon, A. (1992) Reading from paper versus screens: a critical review of the empirical literature Ergonomics: 3rd Special Issue on Cognitive Ergonomics, 35(10) 1297-1326.

Dillon, A. (1996) Myths, misconceptions and an alternative view of information usage and the electronic medium. In: Rouet, J. (et al) (eds.) Hypertext and Cognition, Mahwah NJ: LEA. 25-42.

Dillon, A. and Watson, C. (1996) User analysis HCI-the historical lessons from individual differences research. International Journal of Human-Computer Studies. 45(6) 619-638.

Dryden, L. M. (1994). Literature, Student-Centered Classrooms, and Hypermedia Environments. In C. Selfe and S. Hilligoss (eds.)Literacy and Computers: The Complications of Teaching and Learning with Technology. New York: The Modern Language Association of America. 282-304

Eysenck, M (1990) Personality and Cognition. Entry in M. Eysenck (ed.) Blackwell Dictionary of Cognitive Psychology. Oxford: Blackwell, 267-271.

Frey, D. and Michael Simonson. 1993. Assessment of Cognitive Style to Examine Students' Use of Hypermedia Within Historic Costume. Home Economics Research Journal. 21(4) 403-421

Higgins, K. and Boone, R. (1990). Hypertext Computer Study Guides and the Social Studies Achievement of Students with Learning Disabilities, Remedial Students, and Regular Education Students. Journal of Learning Disabilities. 23(9) 529-540

Hooper, S and Hannafin, M. (1988). Learning the ROPES of Instructional Design: Guidelines for Emerging Interactive Technologies. Educational Technology. July, 14-17

Horton, S., Boone, R. and Lovitt, T. (1990). Teaching social studies to learning disabled high school students: effects of a hypertext study guide. British Journal of Educational Technology. 21(2) 118-131

Jacobson, M. and Spiro, R. (1993). Hypertext Learning Environments, Cognitive Flexibility, and the Transfer of Complex Knowledge: An Empirical Investigation. Technical Report No. 573. Center for the Study of Reading. Champaign: University of Illinois. ERIC Document.

Jacobson, Michael J. (1994) Issues in hypertext and hypermedia research: toward a framework for linking theory-to-design. Journal of Educational Multimedia and Hypermedia. 3(2) 141-154

Jonassen, D. (1989) Hypertext/Hypermedia. Englewood Cliffs: Ed. Tech Publications.

Jonassen, D. and Wang, S. (1993). Acquiring Structural Knowledge from Semantically Structured Hypertext. Journal of Computer-Based Instruction. 20(1) 1-8

Kahn, P., Peters, R. and Landow, G. (1995) Three fundamental elements of visual rhetoric in hypertext. In: W. Schuler, J. Hannemann, and N. Streitz (eds.) Designing User Interfaces for Hypermedia, Berlin: Springer, 167-178

Landow, G. (1992). Hypertext: The Convergence of Contemporary Critical Theory and Technology. Baltimore: Johns Hopkins University Press

Landow, G. and Delany, P. (1991). Hypertext, Hypermedia and Literary Studies: The State of the Art. In P. Delaney and G. Landow (eds.)Hypermedia and Literary Studies. 3-50

Landauer, T. (1995) The Trouble with Computers, Cambridge: MIT Press.

Landauer, T. Egan, D, Remde, J., Lesk, M., Lochbaum, C. and Ketchum (1993) Enhancing the usability of text through computer delivery and formative evaluation: The SuperBook Project. In C. McKnight, A. Dillon and J. Richardson (eds.) Hypertext: A Psychological Perspective, London: Ellis Horwood, 71-136.

Lanza, A. and Roselli, T. (1991). Effects of the Hypertextual Approach verses the Structured Approach on Active and Passive Learners. Journal of Computer-Based Instruction. 18(2) 48-50

Lee, Y. and Lehman, J. (1993). Instructional Cueing in Hypermedia: A Study with Active and Passive Learners. Journal of Educational Multimedia and Hypermedia. 2(1) 25-37.

Lehto, M., Zhu, W. and Carpenter, B. (1995). The Relative Effectiveness of Hypertext and Text. International Journal of Human-Computer Interaction. 7(4) 293-313.

Lin, C, and Davidson, G. (1994). Effects of Linking Structure and Cognitive Style on Students' Performance and Attitude in a Computer-Based Hypertext Environment. In Proceedings of Selected Research and Development Presentations at the 1994 National Convention of the Association for Educational Communications and Technology. Sponsored by the Research and Theory Division (16th, Nashville TN February 16-20)

Liu, M. and Reed, M. (1994). The Relationship between the Learning Strategies and Learning Styles in a Hypermedia Environment. Paper Presented at the Annual Conference of the Association for Educational Communications and Technology and the Association for the Development of Computer-Based Instructional Systems (Nashville, TN February 16-20)

Marchionini, G. and Crane, G. (1994) Evaluating Hypermedia and Learning: Methods and Results from the Perseus Project. ACM Transactions on Information Systems. 12(1) 5-34

McGrath, D. (1992) Hypertext, CAI, Paper or Programs Control: Do Learners Benefit from Choices? Journal of Research on Computing in Education. 24(4) 513-531

McKnight, C., Dillon, A. and Richardson, J. (1991) Hypertext in Context, Cambridge: Cambridge University Press.

McKnight, C., Dillon, A., Richardson, J., Haraldsson, H. and Spinks, R. (1992) Information access in different media: an experimental comparison. In E.J. Lovesey (ed.) Contemporary Ergonomics 1992, London: Taylor and Francis, 515-519.

McKnight, C., Dillon, A., and Richardson, J.(1996) User Centered Design of Hypertext and Hypermedia for Education. In: D. Jonassen (ed.) Handbook of Research on Educational Communications and Technology. New York: Macmillan, 622-633

Nielsen, J. (1995) Multimedia and Hypertextt: The Internet and Beyond. Cambridge MA: Academic Press Professional.

Psotka, J., Kerst, S. and Westermann, T. (1993). The Use of Hypertext and Sensory-Level Supports for Visual Learning of Aircraft Names and Shapes. Behavior Research Methods. 25(2) 168-172

Quade, A. (1993) An Assessment of the Effectiveness of a Hypertext Instructional Delivery System when Compared to a Traditional CAI Tutorial. IN Proceedings of Selected Research and Development Presentations at the Convention of the Association for Educational Communications and Technology. Sponsored by the Research and Theory Division(15th, New Orleans, LA, January 13-17

Recker, M. and Pirolli, P. (1995). Modeling Individual Differences in Student's Learning Strategies. The Journal of the Learning Sciences. 4(1) 1-38

Relan, A. (1991). The Desktop Environment in Computer-Based Instruction: Cognitive Foundations and Implications for Instructional Design. Educational Technology. January 7-14

Repman, J., Willer, H., and Lan, W. (1993). The Impact of Social Context on Learning in Hypermedia-Based Instruction. Journal of Educational Multimedia and Hypermedia. 2(3) 283-298

Rouet, J., Levonen, J., Dillon, A. and Spiro, R. (1996) (eds.) Hypertext and Cognition. Mahwah NJ: LEA

Saga, H. (1992). Are We Ready Enough to Learn from Interactive Multimedia? Educational Media International. 29(3) 181-188

Shin, E., Schallert, D., and Savenye, C. (1994) Effects of Learner Control, Advisement, and Prior Knowledge on Young Students' Learning in a Hypertext Environment. Educational Technology Research and Development. 42(1) 33-46

Shute, Valerie J. (1993). A comparison of Learning Environments: All That Glitters, In: Computers as Cognitive Tools edited by Susanne P. Lajoie and Sharon J. Derry. Hillsdale, NJ: Lawrence Erlbaum Associates. 47-73.

Smith, C. (1994) Hypertextual Thinking. In C, Selfe and S. Hilligoss (eds.) Literacy and Computers: The Complications of Teaching and Learning with Technology New York: The Modern Language Association of America. 164-281

Stanton, N., and Baber, C. (1992). An Investigation of Styles and Strategies in Self-Directed Learning. Journal of Educational Multimedia and Hypermedia. 1(2) 147-167

Tripp, S. and Roby, W. (1990). Orientation and disorientation in a Hypertext Lexicon. Journal of Computer-Based Instruction. 17(4) 120-124

van der Berg, Sjef and James H. Watt (1991). Effects of Educational Setting on Student Responses to Structured hypertext. Journal of Computer Based Instruction. 18(4) 118-124

van Dijk, T.A. and Kintsch, W. (1983) Strategies of Discourse Comprehension. London:Academic Press

Welsh, Thomas M., Kelly P. Murphy, Thomas M. Duffy, and David A. Goodrum (1993). Accessing Elaborations on Core Information in a Hypermedia Environment. Educational Technology Research and Development. 41(2) 19-34.