Here, I will discuss validity in terms of research design. As we create and design the way we are going to answer our research question, there are threats to internal validity and external validity that must be considered.
Internal validity seeks to isolate the cause and effect relationship. In other words, it is focused on making sure that the manipulation (independent variable) has some effect on the dependent variable. This sheds light on the true relationship between the variables. The primary threats to internal validity are:
- History
- Maturation
- Testing
- Instrumentation
- Statistical regression
- Selection
- Experimental mortality
- Selection-maturation interaction
On the other hand, external validity is focused on the ability to generalize beyond the original study across times, settings, measurements, and persons. Threats include:
- Reactive or interaction effect of testing.
- Interaction effects of selection biases and the experimental variable.
- Reactive effects of experimental arrangements.
- Multiple-treatment interference.
I will discuss different designs in a future post (laboratory, survey, field). But it is relevant to note how some of these research designs might have internal/external validity problems.
In laboratory studies, for example, it is possible to isolate the variables of interest to determine if there is a relationship between the independent and dependent variables. This means it is high on internal validity. However, because the laboratory is a highly controlled environment, it is difficult to generalize to conditions outside the lab. So, in a lab study it is important to create a balance between internal and external validity. The greater internal validity, the more controlled the lab experiment is, and so the more difficult it is to generalize.
Field studies are the opposite. When introducing a manipulation, it is difficult to show that it was the manipulation and not something else that had an effect on the dependent variable. The external validity in a field study has fewer challenges to overcome than a lab study. It is possible, with careful research design, to generalize to settings beyond the to the original study.
Depending on the goal, internal or external validity may be more important. If the main goal is only to determine the true relationship between the independent and dependent variables, external validity is not necessarily important. If the main goal is to apply the findings to the world beyond the lab then external validity is important. Ultimately, research goals are interested in both, and thus a balance between internal and external validity must be found. It is possible to have a research design that uses a laboratory study to determine the relationship of two variables, and follow up with a field study to test it.
(Adapted from group and course notes)
(Flashcards and other resources here)