Method Variance

I will briefly discuss method variance, as I have mentioned it as a potential problem in previous posts. It is generally understood that research focusing on macro-level questions (such as strategy) rarely use laboratory experiments, and rarely have the problem of common method variance.

However, each time a research question emerges and the research design is created, it is important to remember and know the whole set of approaches available.

Method variance refers to variance that is attributable to the measurement method rather than to the construct of interest (Podsakoff, Mackensie, Lee, and Podsakoff 2003).

Method variance represents a major problem in research design since it can have a substantial impact on the observed relationships between predictor and criterion variables in organizational and behavioral research. Method variance:

• Is one of the main sources of measurement error.
• Threatens the validity of the conclusions about the relationships between measures 
• Provides an alternative explanation for the observed relationships between measures of different constructs that is independent of the one hypothesized
• Yields potentially misleading conclusions

In order to address this problem there are procedure and statistical remedies.

Procedure remedies

• Obtain measures of the predictor and criterion variables from different sources.
• Temporal, proximal, psychological, or methodological separation of measurement
• Protecting respondent anonymity and reducing evaluation apprehension.
• Counterbalancing question order.

Statistical Remedies

• Herman’s single-factor test
• Partial correlation procedures designed to control for method biases.
• Controlling for the effects of a directly measured latent methods factor 
• Controlling for the effects of a single unmeasured latent method factor 
• Use of multiple-method factors to control method variance
• Correlated uniqueness model 

(Adapted from group and course notes)

(Flashcards and other resources here)