Using Computer Simulations to Teach Decision Making in Reading Diagnostic Assessment for Re-mediation
| Abstract
Five cases of real children encountering serious reading difficulties were transformed into computer-simulated diagnostic case studies to provide opportunities for graduate students in reading specialist courses to apply a diagnostic decision-making model in reading diagnostic assessment. These case studies are presented on the Internet, thus they are called WebCases. This article begins by defining reading diagnostic assessment as the foundation for designing, implementing, and evaluating re-mediation for children encountering reading difficulties. It then differentiates diagnostic assessment from classical measurement, assessment, and evaluation, and describes the importance of decision making in reading diagnostic assessment. The design of the WebCases is then presented, and a link to them is provided. The effects on the learning and confidence of graduate students who use the WebCases is described. |
Related Posting from the Archives
|
For a printer-ready version of this article, click here. Return to the Reading Online site by using your browser’s “Back” button.
Why Diagnostic Assessment Is Essential | What Diagnostic Assessment Is and Is Not | Why a Decision-Making Model? | Computer Simulations | Evaluations and Descriptions | Conclusions and Future Directions | References
Why Diagnostic Assessment Is Essential for Re-mediation
Decision making directs all instructional planning, regardless of whether that decision making is child centered and informed by observations of the child, the child’s reading, and the child’s response to instruction, or whether it simply reflects a belief in a specific curriculum, material, or philosophy. We argue, however, that making instructional decisions for children experiencing severe reading difficulties must be child centered and informed by thorough, valid, and reliable observations that result in
Gathering this information is what we call “reading diagnosis” or “reading diagnostic assessment.” The importance of instructional decision making informed by diagnostic assessment was substantiated in a poll of classroom and reading teachers and reading specialists. O’Flahavan, Gambrell, Guthrie, Stahl, Baumann, and Alvermann (1992) found that 68 percent of those surveyed rated “teacher decision making in the classroom” as highly important; 52 percent also rated “teacher development in assessment” as highly important.
Certainly, reading teachers require knowledge and competencies beyond those associated directly with reading diagnosis. These include, for example,
However, it is not possible to focus such knowledge and expertise on re-mediation for children not making sufficient progress during reading instruction without knowledge of the rationale and processes of the diagnostic assessment of reading. Reading diagnostic assessment is an essential component for gaining the information required to design and deliver the most appropriate reading re-mediation for a child experiencing severe difficulties in reading.
What Reading Diagnostic Assessment Is and Is Not
To some, the verb to diagnose means to search for a cause of a reading problem. Nothing could be further from the truth. To such a simplistic view an H.L. Mencken quip aptly applies: “For every complex problem, there is a simple solution -- which is always wrong.”
The view that the purpose of reading diagnosis is to identify a reading problem and find its cause is arcane and harkens back more than half a century to the medical model of reading diagnosis. In the late 1920s, following growth in the use of group intelligence tests and the individually administered Stanford-Binet Intelligence Test (first published in 1916), it became increasingly obvious that most children who had significant reading difficulties had intellectual abilities that far surpassed their reading abilities. A major research focus between 1920 and 1950 was attempting to delineate the cause (or causes) of reading difficulties (the medical model of reading diagnosis). Among the major variables studied were visual acuity, auditory acuity, general physical status, neurological factors, emotional and psychiatric factors, visual perception, and intelligence. Particularly noteworthy from this era are the comprehensive studies of groups of disabled readers by Marion Monroe (1932) and Helen M. Robinson (1946), both at the University of Chicago. Robinson’s study included a review of extant research on each of the hypothesized causes of reading difficulty and an exhaustive evaluation of 30 children with severe reading problems. Each child was seen individually by a pediatrician, otologist, ophthalmologist, psychiatrist, neurologist, clinical or school psychologist, social worker, and special education teacher.
These two major studies and other smaller scale studies provide not one shred of support for attributing reading difficulties to any single cause, even for a given child. By 1950, there were three decades of accumulated research concluding that the medical model of reading diagnosis -- to search for the cause of the reading difficulty -- was invalid.
Reading diagnosis then took a major turn. The medical model was abandoned, and we moved to the intensive instructional intervention model of “a process of gaining a thorough knowledge of a person’s reading performance, strategies, skills, and instructional needs through accurate observations for the purpose of modifying instruction” (Kibby, 1995, online excerpt).
The word diagnosis comes from the Greek: dia, meaning “thorough” or “through”; and gignoskein, meaning “to know.” Drawing on the definition of diagnostic in Webster’s New World Dictionary, reading diagnosis means an active, problem-solving, decision-making process demanding a careful examination and analysis of the facts in an attempt to understand a reading difficulty. The purpose of this knowledge and understanding is to identify those instructional milieux, methods, and materials that are most likely to facilitate the student’s reading growth.
Reading diagnostic assessment is differentiated from measurement, assessment, and evaluation:
Diagnosis or diagnostic assessment is the most sophisticated of these concepts and processes, and forms the foundation of diagnostic decision making. Diagnostic assessment requires measurement, assessment, and summative evaluation, but adds to it the components of diagnostic teaching (designing, trying out, and evaluating specific instructional methods for a specific child) and using information gathered to formulate an instructional program that will maximize the student’s learning to read. Scriven (1967) assigned the term formative evaluation to judging the quality of an instructional program (material, method, etc.) during the process of instructional design (writing materials, creating new methods, etc.).
Nitko (1996) also uses the term diagnostic assessment, and he too says it is the foundation of diagnostic decision making. He argues that diagnosis centers on the question “What learning activities will best adapt to this student’s individual requirements and thereby maximize the student’s opportunities to attain the chosen learning target?” (p. 9). Nitko continues, “Diagnosis implies identifying both the appropriate content and the features of the learning activities in which a student should be engaged to attain the learning target” (p. 9).
To be a reading teacher or clinician, then, means to be able to measure, assess, and evaluate a child’s reading abilities and, through diagnostic teaching, formulate, implement, and evaluate an instructional program that should facilitate her or his most rapid growth in reading. All of this is what we mean by the term reading diagnosis or diagnostic reading assessment. The raison d’être of the diagnostic process is to determine goals, methods, and materials of reading instruction that will speed up the progress of a child who has encountered significant difficulties in reading; there is no other reason for a reading diagnostic assessment.
The culmination of the reading diagnostic process is -- at the very least -- a written report by the diagnostician for other reading and classroom teachers in which the goals, objectives, instructional parameters (e.g., instructional pace, factors related to interest and attention), teaching methods, and materials for the specific child are recommended and described. To help students in our Master of Education Reading Specialist program prepare themselves to write instructional recommendations, they are given a document entitled “Outline for Preparing to Write Instructional Recommendations” (online document).
Why a Model of the Diagnostic Decision-Making Process?
To integrate the knowledge bases and competencies required during reading diagnosis -- that is, to learn about a child, to know the child’s reading strengths and limitations, and to teach the child diagnostically to determine what methods are effective -- teachers and reading clinicians need an organizational framework or conceptual model of the diagnostic decision-making process (Kibby & Barr, 1999; Gil, Vinsonhaler, & Wagner, 1979; Gil, Wagner, & Vinsonhaler, 1979; Sherman, Weinshank, & Brown, 1979; Wagner & Vinsonhaler, 1978). A comprehensive model of the diagnostic process provides “a chart of the process of reading diagnosis” (Barr, 1995, p. vi).
In addition to a model of the diagnostic process, research has also called for more precise language or terminology for communicating about reading difficulties. Gil, Vinsonhaler, and Wagner (1979), Gil, Wagner, and Vinsonhaler (1979), and Kibby and Barr (1999) argue that communication among reading specialists may be clouded by variations in terminology for expressing methods and findings. Terminology and conceptualization are intertwined, of course, so problems in terminology are related to problems in conceptualization. To teach the diagnostic decision-making process one must focus on both conceptualization and language.
In Practical Steps for Informing Literacy Instruction: A Diagnostic Decision-Making Model, a 1995 monograph published by the International Reading Association, Kibby presents a model of the diagnostic decision-making processes. This text also goes to considerable lengths to be precise in defining words and terms used to communicate about diagnosis, re-mediation, and children who experience reading difficulties. It is the diagnostic decision-making model and the language of reading difficulties used in this monograph that form the basis for the computer-simulated reading diagnoses described in the remainder of this article. An overview of the model and the model may be found at the webpage Decision-Making Model for Diagnostic Reading Assessment.
Computer Simulations for Learning the Diagnostic Decision-Making Process
Creating a model of how experienced reading clinicians conceptualize their decision making during the reading diagnostic assessment process and communicate their thinking, findings, and conclusions is one thing; teaching the application of this decision-making model and its concepts and language is a separate task. The use of simulated paper case studies as class assignments or class activities has a long history as an instructional method for helping students learn the diagnostic process (Barr, Blachowitz, & Sadow, 1995; Harris, 1970). In the last decade, there has been increasing support generally for the use of case studies in the preparation of preservice and inservice elementary and secondary classroom teachers (Kinzer & Risko, 1998; Lundeberg, Levin, & Harrington, 1999; McNergney, Ducharme, & Ducharme, 1999; Merseth, 1991; Shulman, 1992; Sudzina, 1999; Sykes & Bird, 1992; Wassermann, 1994). Several studies attest to the usefulness of case studies in developing pedagogical and philosophical knowledge (Barnett, 1998; Lundeberg & Scheurman, 1997; Stephens, 1997; Wilcox & Lanier, 1999). Case-based methodologies have been shown to facilitate teachers’ decision-making abilities (Griffith & Laframboise, 1997; Harrington, Quinn-Leering, & Hodson, 1996; Herman, 1998; Wilcox & Lanier, 1999).
Moje and Wade (1997), however, caution that cases and case discussions must be “carefully crafted” in order to reap the true benefits of this methodology. Introducing students to the reading diagnostic process with such case studies accompanied by direction and instruction from the professor is useful, but has limitations. First, they are extremely time consuming. Even when entire class sessions are given to paper cases, the process is still rushed and individual students are generally not afforded sufficient “think time” -- that is, time to review notes and readings, to reflect on what they know, and to reflect on the case. In addition, in the classroom, not every student is given the opportunity to put their ideas into words. And since not every student has the opportunity to think aloud, not every student has an opportunity for the professor to evaluate his or her thinking or compare it to the professor’s model. On top of all this, Vinsonhaler (no date, 1979) found that the diagnostic process is not readily taught to novice clinicians with simulated paper cases.
Authors of current research on enhancing learning via technology argue that multimedia cases are more based in reality than are print-based cases (Kinzer & Risko, 1998). Results of studies that have explored case-based methodology that incorporates technology indicate the addition of technology yields pedagogical benefits by promoting teacher inquiry and reflectivity (Abell, Bryan, & Anderson, 1998; Copeland & Decker, 1996; Hughes, Packard, & Pearson, 1997, 1998 [online document], 2000a, 2000b; Kinzer & Risko; Rowley & Hart, 1996; Stephens, 1997). For example, Hughes, Packard, and Pearson (2000b) found the use of video cases and a hypermedia learning environment promoted teacher reflectivity and analysis by providing access to multiple facets of literacy instruction.
In order to enhance the use of case studies to help teachers in our graduate program understand and apply a diagnostic decision-making model and correctly use the language of reading, we added a technology component intended to promote reflectivity and analysis. Logan Scott (second author of this article) created software for computer simulations of reading diagnostic assessment case studies in accordance with Michael Kibby’s (first author) 1995 diagnostic decision-making model. This program could be accessed via the Web (or CD-ROM), hence the computer simulations were called WebCases. In keeping with the basic problem-solving process of the diagnostic model, for a given computer-simulated case, the teacher identifies needed information, justifies requesting that information, obtains that information, interprets and evaluates it, and continues this process until all relevant information is gained. The simulation then asks the student to synthesize, analyze, and interpret all information and provide conclusions and recommendations. Students may shut down the simulated diagnosis at any time and come back to it later, picking up exactly where they left off.
Computer-simulated reading diagnoses would appear to have many advantages over paper case studies. The graduate students may work at their pace, not that of the instructor or the rest of the class. They may leave the website for a while to consult their notes or textbooks for information to aid their thinking. They may walk away from the computer and reflect at length on a particular issue or problem that perplexes them. If they make a mistake, they are able to reflect on what led them to the error and alter their thinking.
In sum, the graduate students are afforded the opportunity and time to research, reflect, and rethink their decision-making processes. Further, they are able to make mistakes and reorganize thinking and conceptualizations without publicly exposing themselves to a class, which seemingly permits them to take greater risks. Finally, the computer can be programmed to provide help the student may choose to use.
For Whom Are the Model and Computer Simulations Designed?
Assessment and evaluation of learning and instruction are crucial in all classrooms, but use of the diagnostic model and the computer-simulated WebCases requires deep knowledge of reading abilities, familiarity with formal and informal testing and assessment techniques and instruments and with a breadth of reading instructional methods and materials. The model is certainly not beyond the abilities of the regular classroom teacher, but it is the reading specialist or reading teacher whom we expect to have the time during the workday to apply the model in the diagnostic assessment of a child encountering reading difficulties; to have studied reading in-depth at the graduate level; to have knowledge of the full range of reading abilities, assessment, and evaluation methods and instruments; and to have detailed knowledge of instructional techniques and materials. That is the job of reading specialists, and it is for them that we intended the model and the computer-simulated WebCases.
We have found that graduate students make more satisfactory progress in learning to apply diagnostic decision making when they begin with a strong background in word recognition, meaning vocabulary, and reading comprehension. This finding is in keeping with what Horowitz (2002) would call “instructivism,” the balance of content versus process. In order for reading specialist students to apply knowledge in the process of reading diagnostic assessment and evaluation, they must first have some content knowledge to apply. Initially, the WebCases came first in the courses, but students too often did not have enough content knowledge about the theory, instructional methods, and evaluation techniques of word recognition, meaning vocabulary, and reading comprehension to apply the diagnostic decision-making processes. Therefore, before beginning to work on the WebCases, students in the reading diagnostic assessment courses prepare and write instructional recommendations (30 to 50 double-spaced pages) for other cases. These instructional reports are in response to a case study in which the students are presented the results of the reading diagnostic assessment -- without the instructional recommendations for re-mediation, the classroom recommendations, or the recommendations to parents, all of which the students must prepare. These reports for the traditional paper cases parallel the recommendations section of the reports clinicians write after they have conducted a full diagnostic assessment of a real child in the reading center, as described in the “Outline for Preparing to Write Instructional Recommendations.”
Format of the WebCases
Five computer-simulation tutorials are completed. Each is based upon a real child (with written parental permission) diagnosed by a university reading center clinician under the supervision of center staff. Each WebCase presents all information gathered in the original reading diagnosis. Data such as case histories and standardized test scores are presented in written form; other data, such as oral reading, sight vocabulary, and word-analysis abilities, are presented both in written form and as audio files. Future cases will include video. Students may print any page of the tutorial at any time; indeed, they must print the oral reading texts and orally read word lists in order to mark what the child says.
There are two forms of the websites: three cases have a virtual clinician, while two do not. The virtual clinician provides a model to the student at each step in the diagnostic decision-making process. A major purpose of the computer-simulated WebCases is to monitor a graduate student’s thinking and language as she goes through the various stages of the diagnostic process, to confirm the student’s work when correct, and to revise and correct her thinking or wording when not correct; the virtual clinician serves this modeling role. As the student works through each WebCase, all of her work is recorded on her tutorial record -- all requests for information and the order of the requests, the rationale for information requested, interpretations of information the program provided, and the student’s summary and recommendations. The student may revise the record at any time. When each case is completed, the student e-mails this record to the instructor.
All five cases allow the student to revise her tutorial record -- including the first three, which were created specifically to give the student immediate modeling or feedback and the opportunity to rethink her rationale or interpretations. The reader may wonder about the student having this ability to revise, especially when she has benefit of the virtual clinician in the first three WebCases. There is no question that the work submitted by the student contains a mixture of her thinking and the virtual clinician’s (as well as ideas and words from the professor’s lectures and the course readings), but this is the purpose of the WebCases: to provide a forum for the student -- with modeling and opportunity for revision -- to think through the reading diagnostic process and to put this thinking into words. This is a creative process that requires the opportunity for revision of both thinking and words.
In spite of the fact that the WebCases are evaluated and graded, they are viewed first and foremost as an opportunity for graduate students to think through the diagnosis of a reading difficulty and to put that thinking into appropriate language. If this takes the student two, three, or more revisions, that is completely acceptable, as acquiring these abilities is the goal. Developing a coherent statement of rationale and interpretation takes time, modeling, and a chance to revise. We strongly encourage this revision, because we believe that with each revision, the student is correcting or extending her thinking in some manner -- that is, she is beginning to find her own voice.
It is possible, we suppose, for students to “cheat” by skipping those steps in which they try to sort out their own thinking, and simply to go right to what the virtual clinician says and does and then return to the tutorial record to copy or mimic. We have not seen evidence that this has occurred as of yet.
The computer simulations website may be visited at www.gse.buffalo.edu/fas/kibby/reading/start.htm. In the dialog box “I want to start a new case,” type the password guest (no space before or after). Click “start a new” (do not hit the return key), and then click “OK” on the small screen that pops up asking “Are you sure you made the right choice?” This will take you to the page titled “The Diagnosis of Reading Difficulties.” Click the “Click here to begin” arrow. The screen that comes up presents pictures of five children (though each case is an actual child, these are not pictures of those children). Read the description for Diane, and then click “Begin Diagnosis.” This brings you to the “Diagnostic Choices” page for Diane.
There are nine major steps to the computer-simulated reading diagnoses.
1. After accessing the website and selecting the child (in this case, Diane), the first screen seen is the Diagnostic Choices page. This contains a list of information potentially available for a given child: for example, case history (home background, school history, previous reports), several sets of formal test scores, informal test results, and observations from diagnostic teaching. The graduate student selects one of these pieces of information. The tutorial record records this selection. (Those using “guest” as a password are able to make selections and write in the boxes requesting information or interpretation, but these selections and texts are not recorded on a tutorial record.)
2. After the student makes a choice of information she wants, the next screen presents a dialog box in which she writes the rationale for requesting this information. What she writes is placed into her tutorial record (which the student is able to revise at any time).
In the first three cases, if the student requests information that is inappropriate in terms of the prior information gathered, the software stops the student and tells her to go back to the Diagnostic Choices page to select other information. (The reader may try this by clicking any button except “case history,” entering “guest” into the “name” box, and hitting the return key or clicking the “investigate” button. This will take you to a message that tells you this is not information you should be requesting at this time, with a modest hint of the information that should be sought. This page has a link back to the Diagnostic Choices page.)
This message to seek different information could be interpreted to mean there is only one order in which to conduct a diagnostic assessment. This is not the case. While diagnosis should not be viewed as a lock-step process of gaining the same information from every child in the same manner and same order (the “one test fits all” approach), neither should it be a random process in which it does not matter what information is collected or in what order. There is a rationale for the order of conducting a diagnostic assessment -- but that order varies from child to child. The decision-making model is sufficiently flexible that it may be applied to emergent readers as well as high school students.
The model used in the WebCases does demand that the first step in the diagnostic process be determining whether the child is reading at his or her expected level, but after that decision has been made, the order of assessment and evaluation -- though still rational -- is flexible. The order is rational because the diagnostic model is hierarchical regarding most components of reading; therefore, certain information should be sought or requested only on the basis of previously gained information. For example, if a fourth grader is able to read 400 words, there is no sense determining if the child has the ability to learn to read words (despite the fact that he should be able to read thousands). Obviously he can read words -- he has learned 400. Therefore, if a graduate student wants to know if the child can learn to read sight vocabulary when taught, the computer simulation will tell that student that she should first determine if the child has any sight vocabulary at all. Similarly, if a child can comprehend a variety of fourth-grade texts, there is no need to determine if that child has difficulty with word meanings in those texts. If the student asks for information about meaning vocabulary, the computer simulation will tell the student that given the excellent comprehension on fourth-grade texts, one can legitimately assume the reader knows the meaning of most of the important words in the passages.
Such directions can be interpreted as suggesting there are right and wrong ways to proceed, but they are in keeping with the foundations of the model -- that is, to understand the nature of superordinate and subordinate skills and strategies in reading, to use time efficiently by assessing only as much as is necessary to gain a full understanding of what the child knows and does not know about reading, to ascertain what she needs to learn to be able to read better, and to determine the form of instruction that does facilitate the child’s learning. A major purpose of the diagnostic decision-making model is to help reading specialists gain a conceptual organization of the components of reading and reading instruction, an organization with some hierarchical structures. In addition, the model and computer simulations are intended to help students learn to use time efficiently in the diagnostic assessment process.
If the information requested is appropriate, the student is then presented a dialog box into which she writes her rationale for requesting the information; this rationale is recorded on the tutorial record. The writing of this rationale is important, and we view it as the think-aloud stage. It is here the student puts into specific terms her first arguments in the diagnostic problem-solving process -- that is, she must interpret the information she already has to determine what information she needs next and to justify her requesting of that information. (Starting from the Diagnostic Choices page for Diane, the reader may try this by repeating the previous directions, but clicking the button in front of the “case history.”)
3. The next page (headed by “The clinician in the actual case”) has two parts:
This is yet another very important stage. We call it the “modeling stage,” where the novice reading teacher is able to juxtapose her thinking and word use with an expert’s -- the virtual clinician. Here, if the student’s thinking, terminology, and writing are on track, she will be reassured to find her rationale matching the virtual clinician’s. When the student finds that her thinking or words do not measure up to the virtual clinician’s, she has the opportunity to return to her tutorial record to change her rationale, her language, or both. It is at this stage that the student begins to develop the ability to think like a reading clinician.
4. After the student clicks “Click here for the case history,” the next screen presents the information requested in written form, orally, or both. (Versions in preparation have video.) The student now applies her knowledge of reading and children to interpreting the information given. This could be viewed as a knowledge application stage. At the bottom of this page is another dialog box in which the graduate student writes her interpretation. She then clicks “Add to Record,” and the next page is presented.
5. Next is a two-part screen, again restating the graduate student’s writing and providing the virtual clinician’s written interpretation of the diagnostic information. If after comparing the two the student feels it necessary, she is able to return to her tutorial record to change her interpretation. As the diagnostic process progresses, each new piece of information must be interpreted in light of previous information.
6. The graduate student is now returned to the Diagnostic Choices page, where she must select additional information, test results, or results of diagnostic teaching. The process of selecting information, providing a rationale for that request, juxtaposing her rationale with the virtual clinician’s, revising her rationale (if she thinks it necessary), receiving and interpreting the requested information, comparing her interpretations to the virtual clinician’s interpretations, revising her interpretations (if necessary), and then going back for more information continues until the student decides all relevant information has been gained.
7. After the graduate student thinks she has all the information she needs, she returns to the Diagnostic Choices page and selects “Final Report.” Now her entire tutorial record is displayed for her to use in writing a summary. Here she edits her tutorial record; summarizes, analyzes, and interprets all the diagnostic information; makes and defends certain decisions; and writes brief recommendations.
8. The graduate student then clicks a button that e-mails the entire tutorial record to the instructor.
9. The instructor reads the record, comments where needed, grades the work, and returns it to the graduate student before class. In class, the case is reviewed.
Evaluations and Descriptions of the Computer-Simulated Diagnoses
The purpose of this article is to discuss the role of diagnostic assessment in reading instruction for children encountering difficulties in reading and to describe a technology-based approach that has been found extremely useful in helping teachers in a graduate reading program learn the rationale and procedures of diagnostic assessment. Observations and evaluations of these graduate students using these WebCases are reported here. These include
How Students Completed WebCases
There are three pertinent aspects of students’ work on the WebCases. First, and contrary to what one might think, for most students, the computer simulations enhanced peer interaction and, presumably, learning from peers. Because students had one or two weeks for each case and grading was not done on a curve, students could help one another -- without risk of compromising their own grades. Students were often in contact by telephone or e-mail, and in many cases, they worked on the WebCases in pairs or trios in a computer lab or on the reading center’s computers. (They did not generally work on a component as a group, but worked independently side by side, sharing their knowledge and questions.) They reported that they frequently wrote out their responses and then shared them, often going back and revising their work after receiving comments from their partners. When we designed the first WebCase, we thought that each student would toil in the evening on a home computer or at the close of the teaching day on a school computer. Although there were students who worked in this manner, most communicated with one another extensively about each case, or worked with at least one other person.
A second relevant and unforeseen outcome was that the students printed nearly every page of each WebCase. Third, students distributed the 6 to 12 hours they spent on each case not over one or two sittings as we expected, but most often over six or more sittings. Students reported that they needed time to find and read content related to data the WebCase provided, to organize their analyses and interpretations of information given, and to prepare their rationale for information they were going to request. They reported that the printouts not only allowed them to read and reread information, but also guided them in reviewing class lectures and discussions, handouts, and assigned readings. This meant that students time and again were going back to texts, notes, and handouts to help them think through a problem or to find particular words to express their thoughts. The WebCases brought about active reading.
Survey Results
In terms of learning gained from them, graduate students in three classes gave the computer-simulated WebCases generally high marks. Approximately 57 percent reported that they learned “a great deal more” from the WebCases than from paper cases, and 40 percent reported they learned “considerably more”; only 3 percent thought they learned less. Respectively, 63 percent and 28 percent reported they thought WebCases improved their learning “a great deal” or “considerably”; 8 percent felt the WebCases did not help their learning.
The computer simulations were designed not only to facilitate cognitive learning, but also to help students gain confidence in their diagnostic decision-making abilities. Though students rated the simulations very high on the cognitive domains in their survey responses, they were less positive about the effects of the WebCases on building their confidence. When asked if the WebCase experience improved their confidence as a reading diagnostician, only 35 percent reported it did “a great deal” and 49 percent said it did “considerably”; 16 percent reported it did not help build confidence at all. However, in spite of what the students reported on the survey, they were actually found by clinical supervisors to demonstrate considerable confidence in planning and conducting their first reading diagnostic assessment of a real child; this point will be elaborated in the next section.
Preparing and Conducting a Real Reading Diagnostic Assessment
The course instructor and clinical practicum supervisors are in agreement: compared to students who do not complete the computer simulations, students who complete the WebCases approach a real diagnostic assessment more systematically, rarely spend time on issues unrelated to reading, and write their analyses and interpretations in clinical reports more clearly and with better organization. An important unanticipated outcome is that these students are also more adept at handling such technical aspects as finding the correct page of a test, giving test directions, recording responses, and other mundane -- but important -- tasks. This means they are better able to attend to the important aspects of a reading diagnostic assessment.
Our observations and discussions with students lead us to conclude that stronger, more organized knowledge of the process of reading diagnostic assessment provided through a well-understood model of the diagnostic thinking process helps novice reading teachers gain confidence about what information they need and what they will do with it. This allows them the freedom to concentrate their attention on the technical requirements of obtaining the information needed from interviews, observations, formal testing, informal testing, and diagnostic teaching. This concentration is necessary for only a brief time in order for the student to become comfortable. Then, by knowing the diagnostic process and handling the tools and techniques of diagnostic assessment, they are quickly able to fix their attention fully on the child.
The simulations have helped the graduate students become more competent and confident in their diagnostic assessments of real children. Although students reported that the WebCases did not have as much impact on their confidence as they did on their learning, clinical supervision staff reported that these students generally approached and conducted diagnoses of real children both more competently and more confidently.
Conclusions and Future Directions
The results from several semesters’ use of these computer-simulated reading diagnostic assessments have been positive. The use of WebCases appears to facilitate group interaction, thus presumably enhancing peer learning; they promote not only reading and rereading, but more active rereading for specific information; and they promote review and distributed learning as students shut down the WebCases to read and think, before coming back to the computer. Students report the WebCases facilitate learning, and supervisors report that they develop student confidence and competence in actual reading diagnostic assessment.
At present, in addition to adding video components, we are attempting to build a culminating WebCase whose Diagnostic Choices page is not a list of potentially available information, but is instead a dialog box. There, the student will request the information she wants, without prompts. For example, if the clinician decides she needs to hear a child read orally, she will request this information with appropriate terminology (e.g., oral reading, miscue analysis, unprepared oral reading). The computer simulation will be prepared to respond to many different terms or words with specific forms of information.
References
Abell, S.K., Bryan, L.A., & Anderson, M.A. (1998). Investigating preservice elementary science teacher reflective thinking using integrated media case-based instruction in elementary science teacher preparation. Science Teacher Education, 82(4), 491-510.
Back
Barnett, C.S. (1998). Mathematics teaching cases as a catalyst for informed strategic inquiry. Teaching and Teacher Education, 14(1), 81-93.
Back
Barr, R. (1995). Foreword. In M.W. Kibby, Practical steps for informing literacy instruction: A diagnostic decision-making model. Newark, DE: International Reading Association.
Back
Barr, R., Blachowitz, C.L.Z., & Sadow, M.W. (1995). Reading diagnosis for teachers: An instructional approach (3rd ed.). New York: Longman.
Back
Copeland, W.D., & Decker, D.L. (1996). Video cases and the development of meaning making in preservice teachers. Teaching and Teacher Education, 12, 467-481.
Back
Gil, D., Vinsonhaler, J.F., & Wagner, C.C. (1979). Studies of clinical problem solving behavior in reading diagnosis (Research Series No. 42). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Gil, D.,Wagner, C.C., & Vinsonhaler, J.F. (1979). Simulating the problem solving of reading clinicians (Research Series No. 30). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Griffith, P.L. & Laframboise, K. (1997). The structures and patterns of case method talk: What our students taught us. Action in Teacher Education, 18, 10-22.
Back
Harrington, H.L., Quinn-Leering, K., & Hodson, L. (1996). Written case analyses and critical reflection. Teaching and Teacher Education, 12, 25-37.
Back
Harris, A.J. (1970). Casebook on reading disability. New York: David McKay.
Back
Herman, W.E. (1998). Promoting pedagogical reasoning as preservice teachers analyze case vignettes. Journal of Teacher Education, 53(5), 391-397.
Back
Horowitz, P. (2002, May). Instructivism: Between CAI and microworlds. Paper presented at the National Science Foundation, REC Principal Investigators’ Meeting, Washington, D.C.
Back
Hughes, J.E., Packard, B.W., & Pearson, P.D. (1997). Reading classroom explorer: Visiting classrooms via hypermedia. In C.K. Kinzer, K.A. Hinchman, & D.J. Leu, Jr. (Eds.), Inquiries in literacy theory and practice (46th yearbook of the National Reading Conference, pp. 494-506). Chicago: National Reading Conference.
Back
Hughes, J.E., Packard, B.W., & Pearson, P.D. (1998, February). Reading classroom explorer: Navigating and conceptualizing a hypermedia environment. Reading Online. Available: www.readingonline.org/past/past_index.asp?HREF=/research/explorer/index.html
Back
Hughes, J.E., Packard, B.W., & Pearson, P.D. (2000a). The role of hypermedia cases on preservice teachers’ views of reading instruction. Action in Teacher Education, 22(2), 24-38.
Back
Hughes, J.E., Packard, B.W., & Pearson, P.D. (2000b). Preservice teachers’ perceptions of using hypermedia and video to examine the nature of literacy instruction. Journal of Literacy Research, 32(4), 599-629.
Back
Kibby, M.W. (1995). Practical steps for informing literacy instruction: A diagnostic decision-making model. Newark, DE: International Reading Association.
Back
Kibby, M.W., & Barr, R. (1999). The education of reading clinicians. In D.H. Evenson & P. Mosenthal (Eds.), Reconsidering the role of reading clinics in a new age of literacy. Greenwich, CT: JAI Press.
Back
Kinzer, C.K., & Risko, V.J. (1998). Multimedia and enhanced learning: Transforming preservice education. In D. Reinking, M.C. McKenna, L.D. Labbo, & R.D. Kieffer (Eds.), Handbook of literacy and technology: Transformations in a post-typographic world. Mahwah, NJ: Erlbaum.
Back
Lundeberg, M.A., & Scheurman, G. (1997). Looking twice means seeing more: Developing pedagogical knowledge through case analysis. Teaching and Teacher Education, 13, 783-797.
Back
Lundeberg, M.A., Levin, B.B., & Harrington, H.L. (1999). Who learns from cases and how? The research base for teaching and learning with cases. Mahwah, NJ: Erlbaum.
Back
McCormick, S. (1995). Instructing students who have literacy problems. Columbus, OH: Merrill.
Back
McNergney, R.F., Ducharme, E.R., & Ducharme, M.K. (1999). Educating for democracy: Case-method teaching and learning. Mahwah, NJ: Erlbaum.
Back
Merseth, K. (1991). The case for cases in teacher education. Washington, DC: American Association of Colleges of Teacher Education and American Association of Higher Education.
Back
Moje, E.B., & Wade, S.E. (1997). What case discussions reveal about teacher thinking. Teaching and Teacher Education, 13, 691-712.
Back
Monroe, M. (1932). Children who cannot read. Chicago: University of Chicago Press.
Back
Nitko, A.J. (1996). Educational assessment of students (2nd ed.). Englewood Cliffs, NJ: Merrill.
Back
O’Flahavan, J., Gambrell, L.B., Guthrie, J., Stahl, S.S., Baumann, J.F., & Alvermann, D.E. (1992). Poll results guide activities of research center. Reading Today, 10(1), 12.
Back
Robinson, H.M. (1946). Why pupils fail in reading. Chicago: University of Chicago Press.
Back
Rowley, J.B., & Hart, P.M. (1996). How video case studies can promote reflective dialogue. Educational Leadership, 53, 28-29.
Back
Scriven, M. (1967). The methodology of evaluation (American Educational Research Association, Monograph Series on Curriculum Evaluation, Publication No. 1). Chicago: Rand-McNally.
Back
Sherman, G., Weinshank, A., & Brown, S. (1979). Training reading specialists in diagnosis (Research Series No. 31). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Shulman, J. (1992). Case methods in teacher education. New York: Teachers College Press.
Back
Stephens, L.C. (1997). How video-cases helped preservice teachers begin to situate themselves philosophically. In C.K. Kinzer, K.A. Hinchman, & D.J. Leu, Jr. (Eds.), Inquiries in literacy theory and practice (46th yearbook of the National Reading Conference, pp. 483-493). Chicago: National Reading Conference.
Back
Sudzina, M.R. (1999). Case study applications for teacher education: Cases of teaching and learning in content areas. Boston: Allyn & Bacon.
Back
Sykes, G., & Bird, T. (1992). Teacher education and the case idea. In G. Grant (Ed.), Review of Research in Education (vol. 18, pp. 457-521). Washington, DC: American Educational Research Association.
Back
Vinsonhaler, J.F. (no date). Improving diagnostic reliability in reading through training (Research Series No. 118). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Vinsonhaler, J.F. (1979). The consistency of reading diagnosis (Research Series No. 28). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Wagner, C.C., & Vinsonhaler, J.F. (1978). On the conceptualization of clinical problem solving (Research Series No. 9). East Lansing, MI: Institute for Research on Teaching, Michigan State University.
Back
Wassermann, S. (1994). Introduction to case method teaching: A guide to the galaxy. New York: Teachers College Press.
Back
Wilcox, S., & Lanier, P.E. (1999). Using cases to integrate assessment and instruction. Mathematics Teaching in the Middle School, 4(4), 232-241.
Back
About the Authors
Michael Kibby (e-mail mwkibby@buffalo.edu) is a professor of education and director of the Center for Literacy and Reading Instruction at the University of Buffalo, State University of New York, USA.
Logan Scott (e-mail slogan@buffalo.edu) is director of the Education & Informatics Technology Node at the University of Buffalo.
For a printer-ready version of this article, click here.
Citation: Kibby, M.W., & Scott, L. (2002, October). Using computer simulations to teach decision making in reading diagnostic assessment for re-mediation. Reading Online, 6(3). Available: http://www.readingonline.org/articles/art_index.asp?HREF=kibby/index.html
Reading Online, www.readingonline.org
Posted October 2002
© 2002 International Reading Association, Inc. ISSN 1096-1232