Your textbook offers a very narrow scope of designs by only reviewing single-case designs. We looked at single-case designs last week along with Chapter 6. As you complete the readings for Weeks 7, 8, and 9 you will see that the book has devoted a chapter to each of those designs. To provide you with an introduction to the major designs available, I will supplement the text book this week by focusing on group designs. Next week, we will learn about qualitative approaches.

True Experimental Designs

 It is important for you to understand the variety of methods available to you to help you make informed decisions. The text book focuses only on one area of possible designs you can use in your classroom, school, district, or community settings. Single Case Designs are a valuable tool, however they are not the only tool out there. Group designs are more appropriate if more participants are available to use and if your research questions are related to knowing the effects on a group rather than an individual. Say you want to know how a specific learning strategy affects 3rd grade reading ability. A single case design might be useful in testing the strategy with children who have a specific disability if you only have one or two children in your class or school with that disability. A group design would be useful if you wanted to test the strategy and it's effects on the whole 3rd grade class. If we want to know if it will be useful for all of our students, then a group design is more appropriate.

Remembering from last week, in a single case design, you are manipulating the independent variable in phases over time, all the while collecting information on your dependent variable to measure changes across phases. In essence, you are comparing the mean of many observations taken from one participant over time.

 In the same way, a group design compares the mean of many observations but those observations are taken from groups of participants all at one time, such as a pretest, for instance. Group designs only allow for a snap shot picture of the group as a whole at one given time, rather than looking at data collected over many observations. Instead of comparing the mean of phases, as you would in a single case design, the group designs compare the means of pretests and posttests and in some cases follow up tests.

Just as we began to introduce single case designs using the most primitive AB format, we can think similarly about group designs. The following lesson will focus on a two group comparison which is the corollary to the AB (two phase comparison). However, the group AB comparison has internal validity and can be used as a design, where the single case AB design does not. This is because the group designs employs techniques like random assignment to comparison groups which lessens threats to internal validity. 

Two-Group Comparison

The most basic group design is the two-group comparison. Basically, a two-group comparison involves randomly assigning participants to comparison groups. Typically you want to have at least 30 in each group so that statistical tests can be used with the data and inferences drawn from the results to larger populations. Students have completed projects with as little as 10 to 15 in each group but this is not advised. You will put a lot of work into it and not know very much about your research questions in the end if you don't have a reasonable sample size.

Then, the comparison groups each experience one level of your independent variable. After experiencing the independent variable for a predetermined amount of time, the groups are tested using the measures of your dependent variables. Sometimes you may be interested in knowing long term outcomes. Some researchers test their participants a year or more after they have completed the original study.

Strengths of this most basic Posttest Only Two-Group Comparison is the use of random assignment to comparison groups, sample size, and using a comparison or "control" group. 

The design can be strengthened by

• adding a pretest, prior to the implementation of the independent variable. The pretest is typically the same measures as the posttest.
• matching participants on the pretests and then randomly assigning matched pairs to comparison groups. This forces the two groups to be equal to start. The need for the two groups to be equal to start with is an important idea. Imagine a horse race where the horses could start from wherever they felt like rather than a starting gate. How would you know who won the race if you don't have an accurate measure of when and where they started? The pretest allows for knowing where they started and random assignment using matched pairs based on the pretest outcome allows us to enforce an even start.
• Making sure the treatment intensity is sufficient. If you are going to go to all the trouble to get approval for and conduct a study, you want to make sure you will find something useful. Too often, students want to cut their independent variable short out of convenience. This is the wrong attitude. You should be thinking, how much of the independent variable will be reasonably  needed to produce a measurable effect? There's just no point in conducting the study if you know before hand that you probably wont find an effect because your independent variable wasn't intense enough or implemented long enough. Don't make this mistake.

Lesson Activities

Visit the Library Tutorial on Primary versus Secondary Sources , and complete the online questionnaire to check your learning.  After you complete the quiz, save the results and upload the file to the dropbox in eLearning.  (Note: You can save the results by using the save as command on the file menu in your Internet browser.) If you are unable to save the results from your browser, save the confirmation message from your email program and upload the file to the Week 7 Lesson 1 dropbox.

Random Assignment and Comparison Groups

The components that make the group comparison design a true experimental design is the use of a comparison or control group and the use of random assignment or matched pairs with random assignment to the comparison groups. These are parts of the design that help increase the validity of the findings. Without at least two groups and without random assignment, the design is by definition, not a true experiment and the inferences we can make from the results are limited as is our confidence in the causal relationship between variables.

What you should take from this is the notion that you will use random assignment to groups if possible and you will have comparison groups.

Keeping that in mind, pretend you want to know how a particular learning strategy affects learning gains between boys and girls. How would you randomly assign participants to comparison groups?

You should be stumped by that one. If you're not, read it again.

What happens when you want to assess the differences between naturally occurring groups that you can't randomly assign people to? How could you assign participants to be either males or females?

Answer: You can't.

You can't conduct a true experiment when your comparison groups are naturally occurring groups that cannot be randomly assigned, like gender, ethnicity, SES, occupation, and age. Instead of conducting an experimental design, you can conduct a quasi-experimental design using pre-existing comparison groups without random assignment. The quasi-experimental designs have less validity than true experimental designs, but for some research questions, we just don't have a choice.

Lesson Activities

• Go to the John C. Pace library online and find an empirically based journal article (one that used data to answer a research question) that uses a group design on a topic you are interested in.
  • Identify the main research question and extrapolate the corresponding null and alternative hypothesis statements from that research question.
  • Skip to the Methods section of the article and read about the "dependent variable", "measurement", "instruments", or any other language that might indicate their outcome variable or what they measured.
  •  Identify the variables they measured and the corresponding constructs.
  • Identify the type of data collected. Was it a scale of measurement? If so, which one? What else did they collect?
  • Identify how many comparison groups they used.
  • Identify how they assigned participants to groups or if they used a quasi-experimental design instead.
  • Write the reference for the article, the main research question, hypotheses, variables, constructs, types of data, number of comparison groups, and assignment.
• Post your work in the Week 7 Lesson 2 dropbox in eLearning.

Need Help?: A sure bet to find a group design is to look up educational research that evaluates the effectiveness of a curriculum or intervention with a Large Sample of participants. The easiest articles for to decipher will use a two or three group comparison. Large evaluation studies will likely use a group design (unless they are targeting a low incidence population). For examples of what a group design may look like, please view the Student Sample Paper #2 or #4. Note the identification of the design in the Method section, the larger sample sizes than what you found in the single subject designs, and the use of statistics rather than Figures with line graphs in the results sections as you saw in the single subject articles.

If you need additional help, seek assistance from the library. Call them, complete the remainder of the tutorials online, or go to the library and ask for assistance.

Need Help?  Have a question but can't find the answer? Here are some options:

• Check the list of frequently-asked questions.
• Email Dr. Barry*.

*Remember to include the course name and your name in your message.

To Page 3

Updated on January 3, 2010