Week 3
(Page 2 of 3) 

Library Lessons This Week

The topics of this week's lessons relate to variables and measurements: what they are, what they represent, and how to ensure that they measure what they are intended to measure.

Lesson 1:

Constructs versus Variables and Scales of Measurement The dependent variable, what we've been calling your outcome variable, is what you measure. These lessons will focus on important aspects of the dependent variable and accurate measurement. Another issue that we have for this week is to begin searching for literature. The assignments for this week will help get you started in learning to navigate in the online library. A construct is an idea or abstract concept while a variable is something that is measured or observed. For instance, self esteem is a construct. To measure self esteem, you need to define it in terms that are measurable and/or observable, and there by, creating a variable which we can call self esteem. For your research proposals that you write in subsequent courses, it will serve you better (read as, cause you less stress), if you choose to include variables in your study in which valid and reliable methods for measuring them already exist. For example, there are an ample number of valid and reliable measures of intelligence that exist. One of the best ways to know what measures exist for certain variables of interest is by reading empirical research that includes these variables. The articles that you find for your literature review should provide you with possibilities for how to measure your own variables if the research you find is close enough to the topic in which you are interested. As you begin to think about your research questions and identifying potential variables, however, it might behoove you to ask yourself how you are going to measure the constructs you are thinking about. If you think measuring a construct will be an extremely difficult task then you might consider another construct that can be easily translated into a measurable variable of interest. The information you collect using measures of variables is called data. There are many types of data that can be collected. Data can be qualitative such as journal entries or interview transcripts. Data can also be quantitative such as the rate, duration, interresponse time, latency, percentages of whole or partial intervals, etc. Much of the data you will eventually collect will by based on a scale. The following are scales of measurement: Nominal Data  Nominal data are categories of information that are grouped by name but have no hierarchical order to them (hence 'nominal'). For example, in the variable "gender" there are two categories, "male" and "female." Researchers will often assign the number 1 to all males and the number 2 to all females. Similarly, for the variable "marital status" there are often 3 categories, "married," "divorced," and "single." A researcher might assign the number 1 to "single," the number 2 to "married," and the number 3 to "divorced." Even though you may have assigned a quantitative representation (number) to these qualitative categories, there is no empirical relationship between the numbers used in the nominal scale that actually corresponds to the actual mathematical relationship between the numbers used. The numbers are simply used to identify members of a given category. No arithmetical procedures such as addition, subtraction, multiplication or division can be used with these numbers. the only statistical procedures that may be used are those based on mere counting, such as reporting the number in each category (e.g., there were 24 males and 36 females participating in this study) or expressing the numbers as percentages of the total number of subjects (e.g., 40% of the participants were male and 60% were female). Ordinal Data  The essential requirement for measurement to be considered ordinal is that it has a distinct order to it (hence 'ordinal'). These are categories of information that have names as nominal data does, but they also have a recognized order. For instance, movie rating G, PG, R, NC17, X. These are named categories that also have an order to them. We know that an R rated movie will have more violence in it than a G rated movie, for instance. Numbers assigned are used to indicate the order of the observation (who has more or less of an attribute) and nothing more. For example, in a mile fun run in which the runners are not timed but are only ranked according to the order in which they crossed the finish line, we only know who came in first, second, third, etc. We have no way of knowing how much faster one runner was than another. The timed difference between the 1st and 2nd runner is not necessarily the same time difference between the 2nd and 3rd runner. The statistics appropriate for an ordinal scale are limited as they are for nominal scales. Since the interval between the categories (ranks) is unknown, you cannot use procedures that assume equal intervals  addition, subtraction, etc. Interval Data  Interval scales of measurement order objects or people according to the amount of an attribute they represent but they ALSO establish equal intervals between the units of measure (hence 'interval'). Numbers on an interval scale may be manipulated by addition and subtraction but not by multiplication or division because there is no absolute zero. For example, temperature as measured in Fahrenheit is an interval scale. The difference between 40 and 50 degrees is the same as the difference between 60 and 70 degrees. However, you cannot say that 80 degrees is twice as hot as 40 degrees because there is no true zero on this scale. You can have a temperature that is a negative number. Any statistical procedures based on adding may be used with interval level data along with any procedures appropriate for lower level data (nominal and ordinal) It is important to point out that it has become common practice to treat many educational variables such as classroom tests and grades as if they were interval data, even when the assumption of equal intervals cannot be justified. It would be difficult to argue that the difference in achievement between an F = 0 and D = 1 (on a GPA) represents the same difference in achievement between C = 2 and B = 3. Because this type of data lies somewhere between ordinal and interval it should be treated as ordinal data. Ratio Data  Only a few variables of interest in education and other social sciences are ratio in nature. With a ratio scale it is possible to multiply or divide values by a certain number without changing the properties of the scale. For example, you can multiply 6 feet by 12 to change the unit of measurement from feet to inches, but it does not change the actual length of what was measured. You can say that someone that weighs 50 pounds weighs twice as much as a person who weighs 25 pounds. All types of statistical procedures are appropriate for ratio scale data. Lesson Activities

Lesson 2:

Operational Definitions Operationally defining your variables is a critical step in planning your research. Simply put, an operational definition of a variable includes both a clear description of what the variable "looks like" in behavioral terms as well as a clearly stated means of measuring that variable. For instance, if I want to measure the construct "paying attention" then I need to create specific measurable variables that represent the construct paying attention. One such variable might be "staying on task." There will likely be many variables that could illustrate "paying attention", but we'll just use "staying on task" as an example for now. Once you have your measurable variables, you have to operationally define those variables. The operational definition will include two parts: 1 Clear description: student is staying on task when they are seated at their desk, with face turned toward paper, pencil in hand, and continuously working toward completing the assignment (not drawing or daydreaming or other). 2 Means of measurement: teacher will measure staying on task behavior by recording the number of whole 5 second intervals in which the student exhibits the description above. Then, a percentage of 5 second intervals can be calculated by dividing the number of successful staying on task intervals by the total number of intervals observed. Observations should be done for at least 60 consecutive minutes a day. Many students forget to include both parts of an operational definition. You will not be able to collect data if you don't know exactly what you are looking for AND exactly how to measure it. This is a very important step! I cannot tell you how many people say they want to study the construct "motivation" and will use that term throughout a proposal and in their research questions without ever giving a thought to the idea of the wide array of ways in which motivation is defined in the literature. Motivation may be identified as selfefficacy in one study or internal/external in another. Some see locus of control as motivation while others look toward selfconcept. Do not fall into this trap. It is important to say what you mean and mean what you say. Hopefully, the brief reading, exercise and case studies here will help you think about how you will need to operationalize the variables you choose for your own research. Lesson Activities
Please note: For this course, please upload ALL files to Elearning as Rich Text Format (RTF) rather than word docs. When you save an assignment in Word, you have the option of choosing "Save As" from the file menu. When you do that, you can select the "File Type" in the window that pops up. Please use the drop down menu under "File Type" to choose Rich Text Format (RTF) before you upload any files to Elearning. 
Lesson 3:

Reliability Reliability is the concept of knowing how consistently a measure can assess a variable. There are many different types of reliability like testretest reliability which tells you the agreement between two administrations of the same test over time if everything else is held constant and interrater reliability which tells you the percent agreement between independent observers collecting data using operational definitions, and more. You will have to deal with the idea of reliability in your measures regardless of the design you choose. This section is dedicated to reliability in research that employs behavioral observation to collect data, specifically interrater reliability. If I am the only observer collecting data, how do we know the data I record is not subjective, based on my opinion? In other words, how do we know the data is scientific, systematic, and objective? One place to start is by using operational definitions to define what data I should be recording. This is a very good start, but even with that operational definition, there is sometimes room for interpretation. Was that time spent rubbing his eyes considered off task, for instance? One observer might score that as on task while another might say that it was off task. To make sure that data are reliable, we need a second observer to independently confirm our findings. Interobserver reliability is the percentage of agreement between two independent observers, who use the same operational definitions during the same time frame to collect data. The requirement to establish reliability is to agree more than 80% of the time on at least 33% of the trials. As an example, if I wanted to calculate interobserver agreement for the variable "staying on task" that we defined above, I would collect data using the operational definition every 5 seconds for at least an hour. During at least 20 minutes of that very same hour, a second independent observer (that means they can't see how I'm scoring the behavior), independently records their interpretation of the behavior also using the same operational definition every 5 seconds. Then, I would take the 20 minutes of data that I collected during the same time that the second observer collected their data and I would compare the findings. I would total the number of intervals in which we agreed. Let's pretend we agreed 238 times. Then I would count the total possible number of intervals in the 20 minute period (240 five second intervals in this example). Finally, I would divide the number of agreements (228) by the total possible number of intervals (240) and then multiply by 100 to yield a percentage of interobserver agreement = 95% Established measures of constructs such as a depression inventory or an IQ exam will have established rates of other types of reliability like testretest reliability, and for those of you who use these established measures that do not rely on direct observation, this exercise in calculating interobserver reliability will not be necessary. Many of you will use direct observations as a means of measuring variables. Those of you who do, will need to use interrater reliability to establish the reliability of your direct observation measures. Lesson Activities

Contact the Library Staff 
Need Help? Have a question but can't find the answer? Here are some options:
*Remember to include the course name and your name in your message. 
To Page 1 
Course Syllabus 
Course Schedule 
John C. Pace Library 
Course Resources 
Student Papers 
Course Home Page 
Updated on January 3, 2010 
© 2004 by Leasha Barry. All rights reserved. 