Bias & Confounding
Will this research produce valid findings?
This is probably the most important question to address when designing a research project. But how can you ensure that your research will be valid? Techniques such as blinding and randomisation can enhance validity, but they do not guarantee validity and they may be inappropriate or impractical for your study. So there is no substitute for making sure that you understand how validity may be compromised and design your study accordingly.
Threats to validity
There are broadly three reasons why findings may not be valid-
1) Chance
2) Bias
3) Confounding
Chance
The measurements we make while doing research are nearly always subject to random variation. Determining whether findings are due to chance is a key feature of statistical analysis. Check our statistics links to find out more about hypothesis testing and estimation. The best way to avoid error due to random variation is to ensure your sample size is adequate.
Bias
Whereas chance is caused by random variation, bias is caused by systematic variation. A systematic error in the way we select our patients, measure our outcomes, or analyse our data will lead to results that are inaccurate. There are numerous types of bias that may effect a study. Understanding how bias occurs is more important than remember the names of different types of bias.
Types of bias
These can broadly be divided into three categories -
1) Selection bias
The selection of subjects into your sample or their allocation to treatment group produces a sample that is not representative of the population, or treatment groups that are systematically different. Random selection and random allocation are the keys to avoiding this bias.
2) Measurement bias
Measurement of outcomes is inaccurate. This may be due to inaccuracy in the measurement instrument or bias in the expectations of study participants, carers or researchers. The latter may be addressed by blinding participants, carers or researchers.
3) Analysis bias
The protection against bias created by randomisation will only be maintained if all participants remain in the group to which they were allocated and complete follow up. Participant who change groups, withdraw from the study or are lost to follow up may be systematically different from those who complete the study. Analysis bias can be reduced by maximising follow up and carrying out an intention to treat analysis.
Accuracy and precision
These two terms are often used in an inaccurate or imprecise way!
Random variation (chance) leads to results being imprecise.
Systematic variation (bias) leads to results being inaccurate.
For example, a huge observational study of 1000's of patients may produce results that are precise, but not accurate. Whereas a small, high quality randomised controlled trial may produce results that are accurate but not precise.
Confounding
This is similar to bias and is often confused. However, whereas bias involves error in the measurement of a variable, confounding involves error in the interpretation of what may be an accurate measurement. A classic example of confounding is to interpret the finding that people who carry matches are more likely to develop lung cancer as evidence of an association between carrying matches and lung cancer. Smoking is the confounding factor in this relationship- smokers are more likely to carry matches and they are also more likely to develop lung cancer.
What is a confounder?
A confounder is a factor that is prognostically linked to the outcome of interest and is unevenly distributed between the study groups. A factor is NOT a confounder if it lies on the causal pathway between the variables of interest. For example, the relationship between diet and coronary heart disease may be explained by measuring serum cholesterol level. Cholesterol is not a confounder because it may be the causal link between diet and coronary heart disease.
Known confounders
Dealing with confounding is relatively easy if, as in this case, you know what the likely confounders are. You could stratify your results- i.e. analyse smokers and non smokers separately, or you could use statistical techniques to adjust for confounding.
Unknown confounders
Dealing with unknown confounders is obviously much trickier. There is always a risk that an apparent association between a risk factor, or an intervention, and an outcome is being mediated by an unknown confounder. This is particularly true of observational studies where patients may be selected to one treatment group or another, not according to any explicit criteria, but by some unknown process, such as a care providers 'gut feeling'. The best defence against unknown confounders is randomisation. This ensures that both known and unknown confounders are randomly distributed between treatment groups.
