Eliminating Bias
Human behavior and outcomes are complex, and there are numerous mechanisms that could be affecting your outcomes.
In this section, we are going to point out a few principles that you will need to watch for before conducting your research or collecting data.
Bias
What if your sample is not representative of your population?
What is Bias?
Bias can be defined as:
Any systematic deviation from the truth that affects the conclusions you make based on your data.
If you do not design your project to identify and eliminate sources of bias, you may not be able to make the correct conclusions.
Example:
For example, imagine you are interested in assessing the attitudes and preferences of EMS providers in your state.
To measure this, the EMS director requests that a group of his buddies complete the survey you have developed. Will the results reflect all the providers in your state? Most likely not. This type of bias is called selection bias because it resulted from the selection of a sample that was not representative of your population.
This type of bias is called selection bias because it resulted from the selection of a sample that was not representative of your population. We will discuss other sources of bias and how to overcome them shortly.
Confounding
Outcomes
Human behavior and outcomes are complex, and there are numerous mechanisms that could be affecting your outcomes.
Confounding
For example, suppose you want to compare the hospital outcomes of those who received medication in the prehospital setting to those who did not. You find that the outcomes for those who did not receive medication were better than the outcomes for those who did. However, this does not take into account that those who received medication were likely to have a more severe illness in the first place. This situation is referred to as confounding.
Mixing of Effects
Confounding is a mixing of effects. It can make it look like there is relationship between two variables when there is none, or it can mask a true relationship.
Example:
Suppose you want to look at how your “Buckle Up” injury prevention program affected seat belt use. You compare seat belt use before and after your program was administered. It appears that seat belt use has improved by 5% since the law was passed. However, during this same time period, a law was passed with primary enforcement for improper child safety or seat belt use. It is possible this factor was responsible for the increased seat belt use. The legislation passed is referred to here as a confounder.
Common confounders include:
- Population changes
- Legislation
- Media Awareness
- Environmental Changes
Overcoming Common Sources of Bias & Confounding
You can eliminate or at least reduce sources of bias and confounding by carefully designing your data project or study.
The following outlines some of the major sources of bias and confounding and how to overcome these in your project design.
- Sample Selection
- Outcome and Other Variable Definitions
- Data Quality
- Comparing Two or More Groups
- Intervention Studies
Sample Selection
Your sample should be representative of your target population. See random selection and introduction to sampling for more guidance in this area. If you are collecting data using a survey or questionnaire, you need to evaluate those who did not respond to the survey and make sure that they are not systematically different from those that completed the survey.
Outcome and Other Variable Definitions
You can reduce bias and confounding by making sure your outcome(s) and other variables are specific, objective, and clearly defined.
For example, suppose you want to describe what percentage of motor vehicle crash fatalities were preventable if immediate medical attention were possible. This could vary depending on who is reviewing the crash circumstances and injuries and how they evaluate the available evidence. Your outcome should be standardized so it can be evaluated consistently across everyone in your sample.
Data Quality
If your data are inaccurate or incomplete, you may introduce bias into your evaluation. Plan your data collection carefully and understand the data definitions and values you are using. Provide education and motivation for accurate and complete data to those who are collecting and entering the data. Institute checks to make sure that the data provided or collected are valid. For example, is 0 meaningful for reasonable diastolic blood pressure?
Comparing Two or More Groups
In a comparative study, you want to evaluate how a specific factor affects your outcome. For example, suppose you want to see if sex (male/female) influences whether or not you get in a car crash. Your study finds that more men drivers are in car crashes than women. You may conclude from this that men are poorer drivers than women. However, you must consider that men typically drive more than women. In this case, the total miles driven are a confounding variable that can affect your results and conclusions.
When comparing two groups (intervention vs. control or factor A vs. factor B), the two groups should be as similar as possible in every way except the intervention or the factor of interest. In a comparative study with an intervention, you can make sure that the intervention and control groups are comparable by randomly assigning participants to either the intervention or control group. If there is no intervention, you can either match cases so that they are similar to controls or you can control for important variables that could also be affecting your outcome in your analysis.
Intervention Studies
If you are implementing a program, policy, or other intervention, there are several key considerations in your project design.
Before and after measures
First, you must take a measurement before you institute the new program or intervention. This is called a baseline measurement and describes your outcome. You then measure the same outcome after the intervention to see how your outcome has changed over time.
Control Group
A control group that is as similar as possible to the intervention group strengthens the conclusion that changes in your evaluation measures are due to your program.
Loss to follow up
You should also consider what participants are available at follow-up (post intervention) versus who originally received the intervention. Is your final sample still representative of your population?