Modules

Module Seven    Causal Comparative and Correlational Research
Module 7 Notes 

Causal Comparative and Correlation Research
Differences Between Causal Comparative and Correlation Research 
Types of Data 
Types of Curves 
Central Tendency 
Variability  
The validity of the research study 
Internal Validity  
Twelve Threats to Internal Validity  
Eight Threats to External Validity  

Both causal comparative and correlational research look at existing characteristics of people that can not be manipulated (age, gender, ethnicity) or are not manipulated due to ethical concerns (grade placement, alcohol use, medical treatment). These types of research are particularly useful in studying natural variation. In addition, results from neither of these types of research lead to conclusions of cause. When a researcher has collected all the data and analyzed and summarized it, the conclusions are still descriptive. Cause and effect can NOT be inferred from these types of research studies. 

The difference between the two lies in the purpose for which each is used. 

The primary purpose for causal comparative research is to compare existing groups (men vs women, upper class vs lower class). The question in your text concerning how to categorize people is resolved when it is a matter of self-report. The respondent is asked what is your gender, ethnicity, age, etc., and the researcher assumes that the respondent knows the answers to these questions and that the respondent will answer these questions honestly. Until other methods are created, this is still the best way to identify groups. These groups must be discrete --- this means each individual is clearly in one and only one of the groups.  

The primary purpose for correlational research is to examine the degree (magnitude, strength) and direction of relationships. Correlational research can be used in comparing groups or individuals and can be used for prediction or for simply describing relationships.  The characteristics included in this type of research must be continuous (see Module 6 Notes, Nature of Measurement).

A correlation is the association between scores (or measures) on two or more variables. The correlation coefficient ranges from +1 to -1 with the sign indicating the direction of the relationship and the number indicating the strength of the relationship. When interpreting correlations, be sure to (1) use the names of the variables, (2) explain the direction in words other than positive/negative, and (3) evaluate the strength in words - not numbers. 

Example:  

1) Create a scatterplot  
2) Approximate the regression line  
3) Estimate the correlation 
4) Interpret the correlation  

1) Nominal -- categories; no logical order 

2) Ordinal -- logical order to groups 

3) Interval --  scores have equal intervals between them 

4) Ratio -- scores have equal intervals between them and a true zero point

Frequency Distribution is the systematic arrangement of individual measures from highest to lowest 
 

Normal -- bell shaped 

Skewed -- negative / positive 

Kurtotic -- mesokurtic, leptokurtic, platykurtic 

Bimodal -- two modes, so has two peaks

Mean - average; used with interval or ratio level data 

Median - the point at which 50% of the scores fall below it and 50% of the scores fall above it; used with ordinal level data 

Mode - the score that occurs most frequently used with nominal level data

Know which measures of central tendency are affected by skewness 

Range = the highest score minus the lowest score plus one; used with all levels of data 

Standard deviation and variance = used with interval or ratio level data 
 

Example: Typing Speed per Minute 

1. Determine the level of measurement and whether the variable is discrete or continuous 

2. Create a frequency table 

3. Create a histogram 

4. Describe the shape of the distribution 

5. Find the mode, median, and mean 

6. Identify the best measure of central tendency 

7. Find the range 

8. Identify the best measure of variation 

(1) Did the independent variable really bring about the observed change on the dependent? This is referred to as INTERNAL validity.  

(2) How well do the results generalize beyond the setting of the study? This is referred to as EXTERNAL validity.

What is the change due to?  

Identify rival hypothesis(ses). 

A study is internally valid only to the extent that extraneous variables are controlled / adjusted for. Because extraneous variables occur concomitantly with the treatment, they can become confounded with it.

1. History - events that take place between the pre and post test especially when the dependent variable is a measure of attitudes and values  
use of control group can help assess this and adjust for it researcher should list variables controlled for and these should be the same in each  
group.  

2. Maturation - the physiological and biological development that takes place in an individual over a period of time, especially in longitudinal studies ; use of control group can help assess this and adjust for it  

3. Statistical Regression - regression toward the mean due to imperfect relationship between pre and post test especially when group is homogeneous (low or high) ; use of control group can help assess this and adjust for it  

4. Instrumentation - changes that occur in the instrument or observational procedures (reason for evidence of validity and reliability of instrument); observer bias; use more than one observer and conduct inter-rater reliability  

5. Testing - may effect subsequent test results especially tests of attitude and achievement tests involving memory; use of control group can help assess this and adjust for it  

6. Mortality - loss of subjects; is a threat to sampling bias; use samples greater than 30  

7. Differential Selection - population and sample are arbitrarily defined by researcher; population is group to which the results from the sample will be generalized, therefore, sample must be representative of the population; random sampling from the population. Even when randomization is used, the chance exists that the sample selected may not be representative of the population.  

8. Selection-Maturation Interaction - same as Differential Selection except that maturation is the specific confounding variable; are effects due to age?  

9. Experimental Treatment Diffusion - members of control group may seek access to treatment; this may happen across teachers or students; contact between the two groups should be minimized  

10. Compensatory Rivalry by the Control Group  - John Henry effect: control group performs beyond what could be expected, because they perceive that they are in competition with the treatment group; effects may be due to unusual motivation; can be provoked by teachers  

11. Compensatory Equalization of Treatments  - Administers of the treatment / control may want to equalize what each group receives if one group is getting goods or services that are desirable; if the goods or services effect post test scores then effects of the treatment are obscured.  

12. Resentful Demoralization of the Control Group - opposite of Compensatory Rivalry by the Control Group; control group performs below what could be expected, because they perceive that they are not getting the special favors that the treatment group is getting; effects may be due to unusual lack of motivation; can be provoked by teachers

Refers to the generalizability and representativeness of findings.  

How relevant are the findings to the population beyond the sample?  

1. Explicit description of the experimental treatment - describe treatment in sufficient detail that another researcher could replicate it; if the treatment works, other educators will want to use it; is there sufficient detail so that they can?  

2. Multiple Treatment Interference - separation of effects is difficult; is the treatment effective by itself or only as it interacts with other treatments; only administer one treatment to each subject  

3. Reactive Effects of Experimental Treatment - re: the Hawthorne effect or the placebo effect; because participants get special attention, differences occur due to this special attention and not due to the treatment  

4. Experimenter Effect  - the treatment may be effective simply because the administrator is more effective than whoever is in charge of the control group  

5. Interaction of Testing and Treatment - when testing procedure is unusual or involves deception, perceptual or cognitive restructuring, surprise, stress  
 

[a] Novelty and disruption effects (Interaction of History and Treatment Effects) --treatment may be effective simply because it is novel and when the novelty wears off, it is no longer effective OR treatment may not be effective because it disrupts the students' routine and when it is accepted as part of their routine, it may become effective  

[b] Pretest Sensitization -- pretest becomes a part of the treatment and usually interacts with the treatment and the post test; may need to conduct the experiment with and without a pretest  

[c] Post test Sensitization -- post test becomes a part of the treatment and may interact with the treatment becoming a learning experience in its own right; this has not been studied as an experimental phenomenon  
 
 

6. Measurement of the Dependent Variable - if treatment is conducted in the same format as the measurement of the dependent variable, then this match may be the cause of the effects  

7. Interaction of Selection and Treatment -if the sample being used is particularly vulnerable to the effects of the treatment, the results may be due to this match rather than to the treatment itself form a broad cross-section of subjects 
  
8. Interaction of Time of Measurement and Treatment Effects  - if multiple post tests are given over time OR if one post test is given immediately following treatment OR if one post test is given several weeks or months later, the results could be different for each of these three situations. Researchers should give one post test immediately following treatment and one post test several weeks or months later to assess retention