Critical evaluation: Validity and bias

Last updated: Sunday, September 06, 2015

All possible measures should be used to reduce the risk of bias in a clinical trial, and the methods section of a paper should explain exactly what steps have been taken. For simplicity, the following points describe a trial of placebo versus drug treatment, but they also apply to trials that compare drug treatments (e.g. drug A versus drug B).

  • Selection of volunteers from a patient population should be random. This stops the researcher from choosing their preferred patient population, so affecting the outcome of their trial favourably.

  • Allocation of volunteers to placebo and treatment groups should be concealed from the researcher. This prevents the researcher choosing patients who they think will do best on their study treatment. For example, the early studies of diphtheria vaccine showed that more patients in the vaccine group died compared to placebo. This was because the sickest patients were chosen to receive the vaccine and the healthier patients were given a placebo.

  • Ideally as many people as possible involved in the trial should be 'blind' to whether volunteers are receiving placebo or treatment. The opposite is ‘open-label’, when everyone knows what the volunteer is receiving. ‘Double-blind’ usually means the investigators and the volunteers do not know which arm of the study each volunteer is in. Blinding is not always possible – for example with drugs that cause distinct side effects (e.g. peppermint oil capsules cause rectal burning) or if the treatment has a complicated dosage regime (e.g. warfarin dosed according to INR results). One way around this is to use something called a ‘PROBE’ design: prospective, randomised, open-label, blinded endpoint evaluation where the people doing the evaluation of the endpoints do not know which group the volunteers have been assigned to.

Courtesy of Simon Wills

  • Baseline characteristics of the groups under study should be as similar as possible. This helps to ensure that any effect seen in the treatment group is due to the treatment and not to pre-existing differences between the groups. The demographics of the groups should be described in the paper. If the baseline characteristics of the groups are very similar this can also be used as an indicator that allocation to groups was truly random.

  • Apart from the treatment or placebo, patients should be treated identically during the trial; they should receive the same number of blood tests, X-rays, and clinic appointments.