Critical evaluation: Making the decision

Last updated: Tuesday, July 14, 2015

So once we’ve assessed the validity and usefulness of a clinical trial paper, how do we then put that knowledge into practice and make a decision about the use of the medicine? An effective, agreed process is needed to review the drug, taking into consideration more than just cost. Most areas of the UK are covered by Area Prescribing Committees (APCs) or Medicines Management Committees (MMCs), and their remit includes balancing the potential benefits and harms of different medicines. One way of deciding between different therapies is known by the acronym ‘STEPS’:

Safety: What are the risks for patients? Look at adverse events from trials.

Tolerability: Do patients remain on therapy? Examine the withdrawal rates from trials.

Effectiveness: How effective was the drug in trials? Is it clinically significant?

Price: Not just acquisition cost, but cost of administration equipment or blood tests, for example.

Simplicity: Is the device complicated? Is one drug given orally and a comparator by infusion?

It is important that this decision making process is based upon clear criteria and is documented appropriately. This is especially important now that, in England at least, the NHS Constitution promises patients ‘the right to expect local decisions on funding of other drugs and treatments to be made rationally following a proper consideration of the evidence’. If the local NHS decides not to fund a drug or treatment that the patient and their doctor think would be beneficial for them, then the patient also has the right to have that decision explained to them.

Being practical with critical evaluation

When you’re doing formal critical evaluation ‘for real’ it’s often in response to a request for a medicine to be added to a local formulary. Often these requests are linked to a newly identified need, the potentially inappropriate use of a medicine, a safety issue, or funding. With this in mind, here are some tips on how to save time and make your work a cut above the rest:

1. See what’s already available
You could spend several days retrieving papers and evaluating them, only to find that someone else has already done the work. For example, UKMi has a list of recent new product evaluations from a wide variety of sources, and databases such as Medline and Embase will help you find reviews in journals. NICE Evidence and the Cochrane Library can also be useful. Using someone else’s work does not mean that you have to use it as it stands, but it gives you another point of view and at least helps you with your literature search.

2. Find out what the issues are
The best evaluations pick out the real issues right from the word ‘go’. You could do a wonderful assessment of the evidence to support the efficacy of a new antidepressant only to find that the key issue was its improved safety, and so your work wouldn’t be very relevant. Identifying the issues can save you time too.

 © Crown copyright 2017
3. Involve clinical experts
By an extension to the above point, it is very clear that some evaluations are not done with the ‘inside clinical knowledge’ that can make them really sharp and relevant. Work with clinical pharmacists and specialist doctors by asking them to comment on your work. Remember that with critical evaluation you are aiming to assess ‘usefulness’ and hoping to change or affirm practice: you’re more likely to do this well if you invite experts to work with you.

4. Evaluate, don’t summarise
Regrettably it can happen quite often that evaluations are little more than summaries. What’s the difference? Evaluations look at validity and usefulness: they place the evidence in a clinical context and point out any limitations. They help the reader use the evidence.

5. Think ‘big picture’
Once you understand the issues in clinical practice, you may find that you need to expand your remit. Maybe you have been invited to appraise one drug, but is it the whole drug class that really needs looking at? You may have been asked to evaluate one particular paper, but perhaps you need an audit of current local practice first? Is it an evaluation that’s needed or a clinical guideline, or both?