When Do Computer Graphics Contribute to Early Literacy Learning?

Shelley B. Wepner
Michelle Cotter

A hallmark of the new literacies is the use of computer graphics to tell a story, demonstrate a theory, or support a definition. Graphics use pictorial representations, and can combine with animation, sound, and music to communicate meaning about ideas, content, and concepts. At their best, computer graphics can take the place of words to convey a message. They are a powerful medium for promoting visual literacy because, although similar to other visual media such as television and video, they also provide opportunities for interactivity.

Research supports the claim that computer graphics contribute to students' content area learning (Baskin, 1997; Card, Mackinlay & Shneiderman, 1999; Larkin & Simon, 1987; Zimmerman & Cunningham, 1991). However, we have not found any significant research that supports the usefulness of computer graphics for early literacy learning. We believe that, to better understand the role of computer graphics in students' early literacy learning, we need to study how these graphics function in relation to text. This article offers a functionality framework for assessing the value of computer graphics for early literacy learning, and provides ideas for determining the value of CD-ROM software and websites.

What Are Different Functions of Computer Graphics?

Surprisingly, we have really only looked at superficial characteristics of computer graphics. Guided by questions used with software evaluation forms, we have tended to view computer graphics as ancillary rather than connected to the text. Questions such as "Is the graphic serving the purpose that it should?" "Is the graphic clear?" or "Is it reinforcing students' correct responses or incorrect responses?" have framed our analysis of the value of computer graphics. These types of questions really have not addressed the function of computer graphics in relation to the written and spoken text.

Because the use of graphics in literacy learning is so extensive, a set of standards should be utilized in determining the function or purpose of graphics used for instruction. The framework for identifying the different functions of graphics discussed in this article begins to apply standards for designing and evaluating software and websites. The functions of graphics can be classified in various ways, but we are using the classifications of organic, supplemental and irrelevant (Tao, 1998). All three types of graphics are used to capture the attention of the learner.

Organic Graphics

Organic graphics provide meaning for the learner. They are intrinsic to the spoken and written text in that they allow the reader to construct an understanding of the ideas and words presented. Text that is potentially incomprehensible benefits from the use of organic graphics. For example, a graphical demonstration of the concept of gravity provides visual cues for forming mental images of this physical force. At a more basic level, a graphic of the concept of a beach uses an expanse of sand or pebbles along a body of water to convey this concept. Other types of graphics cannot take the place of an organic graphic. For example, the only graphic that can be used for a cow is a picture of a cow. It cannot be a graphic of a pig, an elephant, or any other type of animal. Notwithstanding variations in color, size, shape, and so on, the graphic will clearly look like a cow to any viewer.

Supplemental Graphics

Supplemental graphics are those that augment but are not necessary for the comprehension process. Charts and graphs are examples of supplemental graphics because they are not essential for understanding the meaning, and are often considered secondary to the written text. Graphics of beach balls, buckets, and shovels, for example, supplement text to support the concept of a beach. A supplemental graphic can be substituted with another type of similar graphic. For example, graphics to supplement an image of the concept of a beach could include beach chairs and blankets rather than beach balls, buckets, and shovels.

Irrelevant Graphics

Irrelevant graphics have no real connection to the text and do not aid comprehension, but they can enhance the visual value of the screen. Any graphic can be used--an example would be an unrelated cartoon at the end of a section of text about a chemistry concept, or a picture of a calculator at the beach. However, sometimes the graphic actually can detract from the comprehension of the written or spoken text.

Table 1 summarizes the three types of graphics.

The function of computer graphics is generally easier to determine for the illustration of content knowledge than for the portrayal of literacy skills such as phonemic awareness or phonics development. As described in the definition of organic graphics, there usually is a general understanding about the presentation of a cow. However, ways to portray sounds for the letter c vary. The use of computer graphics to teach early literacy development and skills provides multiple challenges. For specific concepts, such as different types of animals, the graphic needs to have definite characteristics for it to be understood as a particular animal. For phonemic awareness or phonics development skills, it is more difficult to use graphics to convey the meaning because of the numerous visual representations available. Despite the challenges inherent in using graphics at this level, the function of graphics in early literacy packages and websites needs to be studied because graphics are so essential for communicating the meaning of the written and spoken word.

Illustrations of Different Graphic Functions

The early literacy packages and websites that we studied focused on skill development and reading development, and typically used a combination of organic and supplemental graphics. We did find subtle variations in how computer graphics were used within the same software products and websites. The computer graphics shifted from being organic and supplemental to being irrelevant and unrelated to the text, often taking away from studentsâ ability to connect with the intended meaning.

The electronic material we reviewed is organized into two categories: comprehensive early literacy programs and electronic storybooks.

Comprehensive Early Literacy Programs

Comprehensive early literacy packages use graphics to communicate the spoken and written text. These packages typically are ãtutorialä and ãskill and practice,ä use a combination of activities for letter and word recognition, use narrative reading passages or storybooks to reinforce the skills, and have graduated levels of difficulty.

IntelliTools Reading: Balanced Literacy uses graphics primarily to communicate the text. This early literacy program uses animal themes across nine units with guided reading and word study activities. Each unit concentrates on a different sound using onset and rime. The program uses primarily organic and supplemental graphics connected to the spoken and written text.

Each unitâs opening screen has a graphic of a book which includes the word book. Within the book is a graphic representation of a character for an upcoming story. The graphic of the book functions organically because it communicates what a book looks like. The graphic of the representative character is supplemental because it is not essential for studentsâ comprehension of that particular screen; rather, it serves as a preview about the main character in the upcoming story. Both graphics, though, serve to enhance studentsâ understanding of the text. (See Figure 1.)

In each writing section, as students create sentences that include a specific rime, pictures from that lessonâs book are used to reinforce studentsâ sentence construction. For example, the student sees a cat in a box for the sentence, ãHere is the cat in the box.ä (See Figure 2.)

Figure 1
Use of Organic (the Book) and Supplemental (Bear) Graphics to Introduce a Story

Source: IntelliTools Reading: Balanced Literacy, 2001, Petaluma, CA: IntelliTools.
Reproduced by permission.

Figure 2
Use of Organic Graphics to Illustrate the Meaning of the Text

Source: IntelliTools Reading: Balanced Literacy (Lesson 9, Unit 1), 2001, Petaluma, CA: IntelliTools.
Reproduced by permission.

Letâs Go Read!2: An Ocean Adventure, the second of a two-part series for developing word recognition skills, uses graphics to supplement the instructional content. As with many programs for developing phonics skills, nonsense graphics related to the theme (in this case, the ocean) are used to narrate, teach, and reinforce. For example, to reinforce the different short vowel sounds, fish inside mailboxes search for postmarked letters that contain specific sounds. Students move the prescribed letter to the fishâs mouth inside the mailbox. Included in the graphic for this activity is a backdrop of ocean artifacts, some of which are irrelevant to the activity because of their lack of connection to the text. (See Figure 3.)

Figure 3
Combination of Supplemental (Mailboxes, Scrolls, and Mailbag) and Irrelevant (Backdrop) Graphics to Teach Vowel Sounds

Source: Letâs Go Read! 2: An Ocean Adventure, 1998, Redmond, WA: Edmark.
Reproduced by permission.

The graphics in these types of packages usually supplement the written and spoken text because of the nature of the content. While most of Letâs Go Readâs graphics support the text, we caution against using other materials that rely too heavily on irrelevant graphics to attract attention. These often distract from the focus on the content rather than enhancing it.

Although we did not find websites that fit our definition of comprehensive early literacy programs, we share two examples that include both phonics games and electronic storybooks. The Sesame Street website uses computer graphics much more functionally for its storybooks than for its phonics games. (See below for a description of the storybook.) While numerous phonemic awareness and phonics games are offered, the graphics do not strongly support the text and are often irrelevant. For example, when students are asked to find hidden letters, the background in which the letters are set could be replaced by any other graphic representation. It is difficult for students to make connections between the graphic and the letter. Graphics would be better used if the hidden letters corresponded with the objects in which they are hidden; for example, have the letter t hidden in a tent.

The Between the Lions website also includes both a “games” section for word recognition as well as a storybook section. (The storybook section is discussed below.) The games section targets phonemic awareness and word construction. This site uses both supplemental and organic graphics. For example, a game called Word Play instructs students to click on the word to see what it does. The word does exactly what the word says; for example, the word pop “pops.” The games also are augmented with spoken dialogue from the characters from the Public Broadcasting Sytem (PBS) television series Between the Lions to add to studentsâ mental image of the narrators.

Electronic Storybooks

Electronic storybooks use graphics to both communicate and enhance the text. These programs use narrative text with characters, plot, and setting to tell a story. They vary in degree of graphic interactivity, based on the level of technical sophistication and expense. For the moment, software storybooks offer more interactivity than website storybooks, but this will most likely change as the Internet evolves.

Three different software storybooks that have high levels of interactivity are the Reader Rabbit series, the Living Book series, and the Big Tuna series. Reader Rabbitâs Developmental Library I offers interactive text that provides two options: automatic text reading (Read to Me) and student-controlled text reading (Read Together). The Read to Me option coordinates the animation of the graphic with the text. The interactivity of the graphics provides additional relevant information and contributes to studentsâ ability to connect the written text to what is read aloud. The graphics are organic because they give the text meaning.

The Read Together option of Reader Rabbit enables the reader to have some control over the graphics. Students choose which animal narrator they want to tell the story. When students choose the classic version (the default story for the graphics used), the graphics serve a supplemental role because of the way in which students use them. For example, when the lion narrates the story of The Three Little Pigs, he introduces the pigs and tells of the beginning of their journey from home. The student-controlled graphics on this screen are directly related to the text of this classic version. When clicked, one graphic shows the pigs saying ãBye, Mamaä as they wave good-bye to their mother. Another depicts the mother crying and waving good-bye to the pigs. However, when students choose alternate narrators to tell the story, the graphicsâ function changes. If students choose to have the wolf read the same story, the same graphics used for the classic version do not match this version. The wolfâs text does not mention the pigs leaving home, yet, when clicked, the graphics still show the pigs saying good-bye to their mother, and the mother crying and waving good-bye to them. Thus, the graphics become irrelevant and distracting. This program demonstrates how the use of the same graphic changes function because of its connection, or lack of connection, to the written and spoken text.

The Living Books and the Big Tuna series of interactive books amplify this. Unlike the Reader Rabbit program, these software programs use authentic literature as a forum for interactive reading. To any viewer unfamiliar with the authorsâ illustrations from their original books, the graphics might seem irrelevant or supplemental. However, because the illustrations are replicas of the authorsâ works, their function is not an issue. As with Reader Rabbit, these series gives students the option to choose if they would like the story read to them (Read to Me) or if they would rather read it with the program (Read Together).

The Read to Me option brings the actual story to life through the animated graphics. The graphics function organically because they help students visualize the meaning of the text by demonstrating the actions of the written words. For example, one ãpageä of Marc Brownâs Arthurâs Birthday shows Arthur running after his friend Francine, to illustrate the text ãArthur ran to catch up with Francine.ä On another page, when the word delivered is used, the graphic shows Arthur and Francine going from house to house to deliver their notes.

In Arthurâs Teacher Trouble, students are shown filing out of the classroom slowly for the phrase, ãThe students filed out slowly.ä On another page, Arthur grumbles for the sentence, ãI got the strictest teacher in the whole world.ä The graphics are used to provide dialogue, actions, and emotions to support the text, thus serving an organic role. An example of a dialogue used to support the text can be found in Mercer Mayerâs Just Me and My Mom. (See Figure 4.)

Figure 4
Combined Use of Spoken Dialogue and Animated Graphics to Supplement Written Text

ãExcuse me, excuse me!- Yes, may I be of service?- We need another chair.- Is someone else joining you for lunch?- Uh-huh. [frog pops up]-That animal is not allowed in this restaurant. The only frogs allowed in here are for our very special frog leg appetizer. [looks of disapproval exchanged throughout the restaurant] - Hey Mom, look! Letâs go get a hot dog for lunch!ä [They leave the fancy restaurant for a hot dog stand outside.] - ã Two hot dogs with the works please. One for me and one for my mom!ä

Source: Just Me and My Mom, 1996, New York, Big Tuna Media Company.
Reproduced with permission

The Read Together portion of the Living Books series, on the other hand, uses many irrelevant graphics. Almost everything on each page is a hot button to trigger animation. For example, in Mark Brownâs Arthurâs Birthday, the reader clicks on a teapot and it spins around with a whirling noise, or the student clicks on the shelf and the door opens with a cat inside. In Arthurâs Teacher Trouble, clicking on pictures on a bulletin board outside of the teacherâs classroom reveals a ballerina dancing, a choir of three animals singing, a cat meowing, and a boy swinging a baseball bat. Although pertinent to the actual bulletin board displays, these graphics are not related to the story line, are not really functional, and most probably are distracting.

In Mercer Mayerâs Just Me and My Mom, the Play option (the equivalent of Read Together) offers animated scenes and graphics that embellish the story. For example, a click on the coffee shop sign in the first screen takes the student to a separate screen in a coffee shop that includes conversations and actions related to coffee shop behavior, but not to the story line. These graphics are supplemental, even irrelevant, to the written text. As with the Read Together portion of the Reader Rabbit and Living Books series, these graphics detract from the storyâs direction.

Website storybooks, while not as sophisticated as the software storybooks just discussed, use graphics to communicate and enhance the written text. There are different levels of interactivity with these website storybooks. The most basic level contains static graphics that simply provide images of the main ideas of the written text. Candlelight Stories offer tales such as ãThumbelinaä and ãRumplestiltskin,ä with images that function to supplement the written text with a graphic representation. For example, a graphic of Thumbelina illustrating her tiny stature is included in the story. These websites are not interactive, but do help to foster comprehension by providing a clear picture of the characters and the text's main ideas.

Another example of this first type of graphic can be found in PBSâs Between the Lions website. The stories from the television series are re-created in text for students to review different lessons. The graphics are merely still clips from each weekâs episode and do little to enhance meaning making. Other websites that have books with static representations of the text are DisneyBooks.com, Lil' Fingers Storybooks, The Pazooter Works, and StoryBox.

The next level of storybook websites contains more dynamic graphics than the previous static ones. These websites have moving images to represent the written text. For example, the Billy Bear Storybooks and the StoryPlace website use animation to give text meaning. When developed correctly, these graphics function organically because they are intrinsic to the written text and cannot be replaced by other types of graphics. For example, one of the StoryPlace stories, ãI Will Not Take a Bath,ä uses animated graphics to place named objects from the text into the bath to communicate reasons that prevent the toddler from taking a bath.

The most interactive website storybooks offer graphics that the students control to communicate and enhance the written text. For example, the Sesame Street story site uses animated graphics both to tell the story and to add depth. Students interact with the graphics to reinforce concepts from the text. In ãElmo Goes to the Doctor,ä animated graphics portray Elmoâs entire experience of going to the doctor. Scenes include the reception area and the examining room of a doctorâs office. The actual graphics used to support the story line are organic because they give the text meaning. Moreover, the graphics of the characters are organic because these characters have an already established representation, similar to Mercer Mayerâs critters and Marc Brownâs aardvark in the Arthur stories. No other representation of the character could take the place of the preconceived one.

When students click on certain graphics to see additional information, they learn more about the purpose of the graphics on the screen. For example, in the reception area, students click the other patients to see what ails them. In the examining room, students click each diagnostic tool to see its function. Because the narratorâs directions for student interaction relate directly to the written and spoken text, the graphics supplement the text. They are not necessary for understanding the story line, yet they do not detract either. Although technically less sophisticated than software storybooks, this example of a website storybook is instructionally more functional than some software storybooks because of its use of graphics to tell a story.

We caution against using website storybooks that are not developed and monitored by respectable educational publishers because they run the risk of using graphics that are both irrelevant and inappropriate. For example, one storybook website we looked at portrays a cat sitting on a rug for the written text about a cat sleeping on a bed.

Final Thoughts

Unquestionably, computer graphics contribute to the reader's ability to form mental images about written and spoken text. However, computer graphicsâ functional relationship to the text determines their real value. Graphics that give text meaning (organic) or augment the meaning of the text (supplemental), whether written or spoken, contribute to studentsâ meaning making.

In the past, more graphics meant a better learning experience, and frequent interaction with computer graphics was meant to enhance the learning experience. But besides being costly, graphics that are irrelevant often promote confusion and are distracting because of a lack of connection with the main thrust of the text.

Animated, dynamic graphics that support student interaction should be used for early literacy development when they function to enrich the text or take the place of the text. Such graphics teach or reinforce concepts and content. When graphics are unrelated to the text, they function to capture attention that may or may not contribute to learning. A functionality framework can assist in analyzing the degree to which graphics support the text and concepts to be learned.

References

Baskin, B.H. (1997). The role of computer graphics in literacy attainment. In J. Flood, S.B. Heath, & D. Lapp (Eds.), Handbook of research on teaching literacy through the communicative and visual arts (pp. 872-874). New York: Simon & Schuster Macmillan.
Back

Card, S., Mackinlay, J.D., & Shneiderman, B. (1999). Readings in information visualization: Using vision to think. San Diego, CA: Academic.
Back

Larkin, J., & Simon, H.A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11(1), 65-99.
Back

Tao, L. (1998, October). Bridging the gap between readers and textbooks: Textbook-reading strategies that facilitate content learning for middle grade students. Paper presented at the 1998 State Conference of the Kentucky Middle School Association, Bowling Green, KY.
Back

Zimmerman, W., & Cunningham, S. (1991). Visualization in teaching and learning in mathematics: A project by the Committee on Computers in Mathematics Education. Washington, DC: Mathematical Association of America.
Back



About the Authors

Shelley B. Wepner is associate dean of the School of Human Service Professions, director of the Center for Education, and professor of education at Widener University (One University Place, Chester, PA, USA; e-mail: shelley.b.wepner@widener.edu). She specializes in the use of technology for literacy development and teacher education and leadership qualities of education deans and reading specialists. Her most recent book publications include the third edition of The Administration and Supervision of Reading Programs (Teachers College Press, 2002) and Linking Literacy and Technology: A Guide for K-8 Classrooms (2000, International Reading Association). She currently is studying the shifting responsibilities of classroom teachers and teacher educators in infusing technology into teaching, including the appropriate use of graphics for literacy learning.

Michelle Cotter is a former graduate assistant at Widener University where she pursued her masterâs degree and certification in Reading and English. She is studying the use of technology for instruction to enhance comprehension for struggling readers. She currently tutors English as a Second Language learners and students with comprehension difficulties. She can be contacted at cotter3@hotmail.com.

Back to top

To print this article, point and click your mouse anywhere on the article's text; then use your browser's print command.

Citation: Wepner, S.B., & Cotter, M. (2002, February). When do computer graphics contribute to early literacy learning? Reading Online, 5(6). Available: http://www.readingonline.org/newliteracies/lit_index.asp?HREF=wepner/index.html