The Formative Experiment
Formative experiments have been used by several literacy researchers in the field of literacy (see, e.g., Jiménez, 1997; Reinking & Pickle, 1993; Reinking & Watkins, 2000; Welch, 2000). This research methodology has developed as a response to the shortcomings of traditional qualitative and quantitative research methods, which have not been able to address satisfactorily two questions that are crucial to instruction, especially in the ever-changing context of information and communications technologies (Reinking & Watkins, p. 384):
The formative research design overcomes these problems as it allows researchers to become actively engaged with the participants and institutions involved in the research and to encourage change (Jiménez, 1997). Jacob (1992) has pointed out that formative experiments aim to improve instruction through the combination of qualitative methods of investigation and interventions in learning situations. The epistemological stance associated with formative experiments is pragmatism (Reinking & Watkins, 2000) -- in other words, data collection, analysis, and interpretation are focused on the pedagogical goal or goals.
There are some similarities between formative research, reflective practice (Henderson, 1992), and diagnostic teaching (Walker, 2000), which many teachers practice as a matter of course. Van Lier (1996) has suggested that reflective practice and academic research can be seen as the two extremes of a continuum, with increased systematic approaches, documentation, and sharpening of questions marking academic research. The formative research carried out in this study was systematic and well documented, and thus distinguishable from teachers’ everyday reflective or diagnostic teaching.
The formative experiment as a research design is still evolving, but has been used successfully in several studies. Jiménez (1997) used this design in order to help teachers improve their practice in teaching five low-literacy Latina and Latino readers in middle school. The formative experiment was selected as the research design because of a desire to go beyond the typical qualitative research foci of observation, interviews, and document analysis, and to become actively involved with the participants in order to bring about change. In this instance, a series of cognitive strategy lessons was carried out, with responses systematically recorded. These responses were then used to shape and modify the experiment in various ways to achieve the pedagogical goal of improving students’ comprehension of text. All five children were reported to have derived some benefit from the cognitive strategy lessons.Reinking and Watkins (2000) used a formative experimental design to find out how the use of multimedia book reviews might increase the independent reading of young students. Diverse qualitative and quantitative data were gathered during two academic years in four fourth-grade and five fifth-grade classrooms in three schools. The first six weeks were spent gathering qualitative data in order to gain a thorough understanding of the students, teachers, classrooms, and schools. Observational field notes and interviews with students and teachers were the main methods of data collection. Quantitative data were also collected to establish a baseline to facilitate comparison of the amount of independent reading done before and after the intervention.
Four students in each classroom were then taught how to write multimedia book reviews using computer software. The students were were not aware that they were the focus of attention. A formative experiment was conducted, with the goal of increasing the children’s independent reading. This was achieved, as were several unintended consequences, which are important to consider in formative experiments. Because data were collected across more than one classroom context, it was also possible for the researchers to observe variations in the effects of the intervention, and to speculate about the mitigating factors involved. Reinking and Watkins concluded that the formative experiment methodology is a potentially valuable means of exploring the use of technology to enhance literacy learning.
Formative research follows a case study approach that accommodates exploratory work and lends itself well to teacher-researcher collaboration. Formative research involves “designed” cases rather than naturalistic ones, as the researcher manipulates the situation under investigation and then “formatively evaluates the instantiation” (Reigeluth & Frith, 1999, p. 637).
It must be noted that there are several varieties of formative research, such as action research and formative evaluation research. These are related and not yet clearly distinguishable (Reinking & Watkins, 2000). Action research can be defined as follows:
Action research is any systematic inquiry conducted by teacher researchers, principals, school counsellors, or any other stakeholders in the teaching/learning environment, to gather information about the ways that their particular schools operate, how they teach, and how well their students learn. This information is gathered with the goal of gaining insight, developing reflective practice, effecting positive change in the school environment (and on educational practices in general), and improving student outcomes and the lives of those involved. (Mills, 2000, p. 4)
Action search is similar to the formative experiment in that it focuses on gathering information with the goal of effecting educational change. It involves identifying an area of focus, collecting data, analyzing and interpreting data, and developing an action plan. However, it differs from the formative experiment in that it is usually conducted by teachers for teachers, and is not imposed by someone else (Mills, 2000). In action research, the teachers themselves choose the area of focus, determine research collection techniques, collect, analyze, and interpret the data, and develop action plans, whereas in a formative experiment they may participate in these activities but they do not enjoy the same degree of autonomy. Furthermore, the results of action research are often not disseminated beyond the school itself.
Formative evaluation is a method used for looking at innovations, educational or otherwise. The information gained is for the developer of the innovation (i.e., the researchers or teachers) to use in order to inform modifications. “The developers introduce the innovation into a suitable context, or a small number of contexts. They then monitor its use to determine how its features work, with the goal being to make appropriate modifications to the innovation” (Bruce & Rubin, 1993, p. 6). This description of formative evaluation research sounds almost indistinguishable from formative experiments. However, formative experiments are broader in their scope than evaluation research and do not necessarily restrict their audience to the developer of the innovation. Formative evaluation is an important component of formative experiments, but it is not the same thing.
In summary, formative experiments in educational settings involve collaboration between teachers and researchers, but the researchers assume most of the responsibility in most of the stages, especially in data analysis and publication. This method, which encourages change in order to accomplish prespecified outcomes, is still evolving and is closely related to others, such as action research and formative evaluation research. It is hoped that, as this method gains popularity, it will continue to evolve and become more clearly distinguishable from the other methods.
Return to the main article text
Section References
Bruce, B.C., & Rubin, A. (1993). Electronic quills: A situated evaluation of using computers for writing in classrooms. Hillsdale, NJ: Erlbaum.
Henderson, J.G. (1992). Reflective teaching: Becoming an inquiring educator. New York: Macmillan.
Jacob, E. (1992). Culture, context, and cognition. In M.D. LeCompte, W.L. Millroy, & J. Preissle (Eds.), The handbook of research in education. San Diego, CA: Academic.
Jiménez, R.T. (1997). The strategic reading abilities and potential of five low-literacy Latina/o readers in middle school. Reading Research Quarterly, 32(3), 224-243. Available: www.ingentaselect.com/ira/00340553/v32n3/contp1-1.htm
Mills, G.E. (2000). Action research: A guide for the teacher researcher. Upper Saddle River, NJ: Merril.
Newman, D. (1990). Opportunities for research on the organizational impact of school computers. Educational Researcher, 19(3), 8-13.
Reigeluth, C.M., & Frick, T.W. (1999). Formative research: A methodology for creating and improving design theories. In C.M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory (Vol. II, pp. 633-651). Hillsdale, NJ: Erlbaum.
Reinking, D., & Pickle, J.M. (1993). Using a formative experiment to study how computers affect reading and writing in classrooms. In C. Kinzer & D.J. Leu (Eds.), Examining central issues in literacy research, theory, and practice (pp. 263-270). Chicago: National Reading Conference.
Reinking, D., & Watkins, J. (2000). A formative experiment investigating the use of multimedia book reviews to increase elementary students’ independent reading. Reading Research Quarterly, 35(3), 384-419. Available: www.ingentaselect.com/ira/00340553/v35n3/contp1-1.htm
van Lier, L. (1996). Interaction in the language classroom: Awareness, autonomy and authenticity. New York: Longman.
Walker, B.J. (2000). Diagnostic teaching of reading: Techniques for instruction (4th ed.). New York: Longman.
Welch, M. (2000). Descriptive analysis of team teaching in two elementary classrooms: A formative experimental approach. Remedial and Special Education, 21(6), 365-376.
Return to the main article text
From Oakley, G. (2003, March). Improving oral reading fluency (and comprehension) through the creation of talking books.
Reading Online, www.readingonline.org
Posted March 2003
© 2003 International Reading Association, Inc. ISSN 1096-1232