Executive Summary

INTRODUCTION
Who is this pack for?
Where does this information come from?
What sorts of complementary medicine does the pack cover?
Which sections should I read?

SECTION 1 - IMPROVING COMPLEMENTARY MEDICINE:
What can clinical audit do?
What does audit do?
Why do audit in complementary medicine?
What can be done with audit in complementary medicine?
Why not audit outcomes?

SECTION 2 - CASE STUDIES: CLINICAL AUDIT IN ACTION
Introduction
Recording patient information: the Initial Consultation
Looking after the tools of the trade: keeping treatment materials safe and effective
Recording and following-up lifestyle advice
Communicating with other health professionals: letter writing
Communicating with patients: individuals who discontinue treatment
Communicating with patients: discussing progress during consultations

SECTION 3 - FURTHER HELP
Information sources
Complementary medicine organisations
References/Further reading
Glossary

APPENDIX 1 - CLINICAL AUDIT IN MORE DETAIL
What needs to be done, when?
How do I decide which topic to choose?
How do I do sample cases?
How can I use questionnaires?

APPENDIX 2 - GUIDANCE FOR INDIVIDUAL PRACTITIONERS
Can clinical audit help me find out if I am effective?
Do I have the skills and technology?
Who can help?

APPENDIX 3 - SUPPORTING AUDIT IN COMPLEMENTARY MEDICINE
Assisting individual therapists do audit
Promoting audit within a profession

Acknowledgements
The following complementary therapists took time from teaching and practice commitments to audit their own work and provide invaluable feedback: Caroline Beatty; Ann Beavis; Richard Booth; Alison Denham; Arnold Desser; Dennis Donnelly; Andrew Flower; Marie McShea; Kalim Mehrabi; John Moorhouse; Jonathan Parsons; Jill Pay; Avril Sanders Royle; Serena Scrine; Ajay Shah; Jackie Shaw. The author would also like to thank the following for their help and contributions: Liz Carroll, the programme's audit facilitator; Sue Lister; Fiona Sharples; Mark Charny; David Pruce; Jonathan Field; Jenny Langworthy; Salma Jeevanjee; Rob McCarney; Jonathan Monckton. The RCCM's audit programme was part funded by the NHS Executive and Henry Smith's Charity.

In doing audit, individual complementary therapists may often need to deal with practical difficulties which other health professionals may not face. In all but a few professions:

• access to academic or peer support and to useful facilities is poor
• the development of clinical guidelines is at a very early stage.

Individual complementary therapists are therefore often likely to need:

• well-timed advice on the scope of resources available for audit
• clear advice at an early stage on the strengths and limitations of the audit approach
• support so as to develop their own, sound criteria for good practice.

A commitment to clinical audit is now a requirement for professional clinical practice in the UK. Several professional bodies in complementary medicine have started work to promote audit. While steps to assure high quality care among individual practitioners have been taken by a small number of professions, guidance on good practice is urgently needed from the less developed therapy bodies.

Many complementary practitioners will already be using the principles of clinical audit in their every day work. It makes sense to make this process more explicit and more systematic - for individual learning, for improved communication between professions and for improved patient care.

Who is this pack for?
The pack is aimed at two different audiences:

• It is an introduction to clinical audit for complementary therapists. It presents basic principles and illustrates these with examples of audit from actual complementary therapy settings.
• It is a reference book for groups and individuals who want to support or promote clinical audit in complementary medicine. The pack contains a list of active conventional and complementary therapy organisations and information on the aspects of audit likely to need the most support.

Where does this information come from?
The pack is based upon the findings of a programme run by the Research Council for Complementary Medicine (RCCM), part funded by the Department of Health. A total of 16 therapists volunteered to attempt audit in their own clinics or professional organisations. The therapists received basic audit training, guidance from an experienced audit facilitator and a small amount of funding. The therapists and the facilitator reported back on their progress after eight months.

What sorts of complementary medicine does the pack cover?
Given the scarcity of written material on audit and complementary medicine, the programme findings should be of interest to most people working in the field. Nonetheless, it is worth recognising that complementary medicine (CM) is an extremely diverse field. A number of recent introductory texts exist (see Further help).

The programme involved registered practitioners of the following therapies: acupuncture, chiropractic, healing, herbal medicine, homoeopathy, osteopathy and reflexology. All but one of the therapists involved in the programme were so called 'independent' therapists - individuals that practice complementary therapy first and foremost, not as part of a wider job remit. An increasing number of doctors, nurses and physiotherapists are also trained in one or more complementary therapy techniques.

Which sections should I read?

• Section 1 explains what clinical audit aims to do and why it is important in modern healthcare.
• Section 2 is a set of case studies based upon real life examples of audit in complementary therapy settings.
• Section 3 lists sources of support and information, along with a glossary of terms.
• Appendix 1 contains practical detail on how to carry out an audit.
• Appendix 2 offers general guidance to practitioners wanting to audit their practice.
• Appendix 3 contains guidance on supporting clinical audit in complementary medicine.

What can clinical audit do?
This section describes the rationale for clinical audit. It looks at what audit can do and why is it important for complementary medicine. For more detail of how to carry out an audit see the case studies in section 2, or Appendix 1.

What does audit do?
The idea behind clinical audit is extremely simple - that patient care can often be improved. Audit is a process that helps change happen. Practitioners draw upon their own expertise to look at how they care for patients and see if they can improve this care by making appropriate changes. 

This process is often summed up by an audit cycle

The audit cycle shows how clinical audit asks the following questions about clinical practice:

1. What am I trying to do? The answer to this will depend on the subject in hand. Where there is research evidence this should be used. Where there is no research, professionals can get together to come up with a consensus.
2. Am I doing it? This part requires measurement - the nearest audit gets to technology.
3. Why am I not doing it? If you have chosen a good audit subject you will usually find you have to ask this question.
4. What can I do to make things better? This requires a systematic approach and planning. Practitioners are used to thinking about their own clinical decisions. They tend to be less used to looking at the way they work with other professionals and with organisations as a whole.
5. Have I made things better? Check whether your action plan has a) been carried out and b) been effective. If it has not, you may need to go back a stage.

An example of audit in practice
A group of herbalists looked at handling answerphone messages left by clients. The group:

• first identified certain criteria for good practice, for example returning calls within 24 hours or ensuring that they alter their outgoing answerphone message if they are away for more than one day.

what am I trying to do?

• then collected data to identify whether their criteria were met or not: this involved keeping a record of a week's calls and how they were handled.

am I doing it?

• then discussed their findings as a group and developed ideas for improving. Most decided to change their outgoing answerphone messages in some way.

why am I not doing it? what can I do to make things better?

• they agreed to reassess six months later.

have I made things better?

• The group reported improved responses to patient enquiries and noted: "focusing on the topic led to an exchange of views and opinions, which in itself was enlivening and useful".

• Did I treat that patient as well as I could have done?
• Was there something else I should have done?
• How can I make changes tomorrow?

It makes sense to make this process more explicit and more systematic, both for individual learning and for improved communication between professions.

Furthermore, a commitment to audit is now a requirement for professional clinical practice in the UK (Department of Health, 1994, 1998)

• the Department of Health's Chief Medical and Nursing Officers have stated, "Clinical Audit should be a routine practice for all health care professionals".
• Under the new NHS framework of clinical governance, GPs and other purchasers of healthcare will have responsibility for the quality of the care they provide. They will therefore need to work with professionals who understand the principles of audit and are prepared to take part in audit projects.

Individual health professionals are now expected to carry out audit as a means of developing their practice. Professional bodies in complementary medicine will need to consider developing audit schemes for their members. Several have started work to promote audit (see Further help). The chiropractic and osteopathy professions have both been involved out centrally guided, profession-wide audits of single topics.

• Standard audit methods work within complementary medicine. The process of audit is as applicable to complementary medicine as it is to other areas of healthcare. The very same procedures used for several years by nurses, doctors and other professionals can be applied.
• Appropriate criteria can be set. Audit criteria can be based upon local or national consensus about professional practice. Localised expertise - about a clinic's patients or communication systems, for example - can be used. National therapy bodies can issue guidance that can then be used to assess and improve practice. Conventional health professions have found that research evidence for interventions is useful if it is present, but is not essential. The important thing is to start audits by answering the question "what am I trying to do?" in a reasoned way.

They described:

seeing improvements in patient care.

"My notes are now more useful to me"

"Discussion is now more focused in my consultations. I am able to bring up more issues"

noting useful changes in their approach to work.

"Audit has helped me think more clearly about what I do".

learning valuable lessons from the process.

"I found I was lacking in team-building, information giving and selling skills"

"I finally learned how to use my computer"

"Audit was a big learning process that took time but produced a quantum leap".

the value of audit in daily practice

"I now see audit as formalising common sense"

"Audit is human scale, real world investigation"

"The project helped to give energy to meetings"

However, this kind of approach is often just not appropriate (see figure 1). Outcomes are potentially the result of many things outside the control of practitioners - and therefore beyond the capabilities of an audit cycle. Outcomes are also difficult to assess. Instead, the argument for auditing process is as follows:

1. we identify outcome x as being important
2. research or experience shows or suggests that outcome x is produced by process y
3. we audit process y.

Recording patient information:The Initial Consultation
Looking after the tools of the trade: keeping treatment materials safe and effective
Recording and following-up lifestyle advice
Communicating with other health professionals:Letter writing
Communicating with patients: individuals who discontinue treatment
Communicating with patients: discussing progress during consultations

The following case studies are based on real audit projects carried out by complementary therapists. They illustrate the flexibility of audit. All projects resulted in better care for patients. All take slightly different routes around an audit cycle.

Look out for the following stages:

1. Defining the problem. Why this area of care is important.
2. Purpose of the audit. This makes the aims of the audit explicit from the start.
3. Criteria. These are definitions of good practice. They can be generated from many sources, including research evidence, professional organisations, the Patients' Charter, peer-group consensus etc.
4. Standards. These show how vital it is that each criterion is met. Some criteria will be more critical than others and so need a higher standard. Setting standards helps prioritise action.
5. Data collection. Tried and tested techniques can investigate how well you are doing. Data may be previously recorded information. Fresh data can be collected by questionnaires or forms. Most audits summarise information using simple data collection sheets. Further information about sampling and questionnaires is given in section 4.
6. Analysis . Having collected information about your practice, it is time to compare your results against the standards set at the start of the audit. This stage asks, "where am I meeting the standards, where not?"
7. Discussion. Discussion with colleagues or peers helps even if they are not directly involved with your audit project. Reassess your criteria and standards and generate ideas about what can be done to improve things. Sometimes an audit of one area will highlight problems in another area. An audit of record keeping, for example, may suggest that more time is needed for a first consultation. Keep talk focused around a specific problem and allow differences in opinions and perceptions. Take time with this stage so that you come up with sound ways forward.
8. Managing Change. The most critical part of the audit comes when you decide what can be improved and how. There is usually at least one aspect of patient care that can be improved. If you are working in a team it is important to reach agreement about what needs to be changed and how. It is equally important to take steps to manage the changes. Agree what will be done, by whom, by what date. Implement the changes and re-audit at an agreed date in the future.

Defining the problem
The collection and recording of information during a patient's first consultation is a fundamental aspect of clinical practice. Certain information needs to be in a patient's notes for reference. Some say that if it isn't written down, it didn't happen. Nevertheless, poor standards of note taking are common across all health professions. Few CM professions have looked into this aspect of practice.

Purpose of this audit.
This audit was carried out by four osteopaths working in a multi-partner clinic. They wanted to:

• see how well they collected and recorded information from patients in a first consultation
• compare practice amongst themselves
• compare this with best practice and improve where necessary

Criteria
Practitioners are taught to collect and record specific clinical information during their undergraduate training. Often techniques are then adapted with experience. The osteopaths referred to documents produced by their professional body and decided that the following should be recorded at a first consultation:

• Personal Details - Name (first and surname), Address, Telephone number, Date of birth, Gender, Occupation, GP name and contact details
• Presenting Complaint - Site, Nature, Date of onset, Causative factors, Duration and progression, Factors affecting symptoms, Past history
• Medical History - Current general health, Medication, Investigations and treatments, Illness/accidents/surgery
• Other - Family medical history, Diagnosis, Treatment plan, Records should also be signed, dated and legible

Standards
The osteopaths decided:

• just over half of their criteria were "critical" and needed targets of 100% (eg. name, address, gender, site and nature of presenting complaint).
• others (eg. GP details and medical history criteria) were considered a lower priority and were therefore be set lower targets of 80 or 90% in this audit round.

Data collection

• The osteopaths in our programme wanted an up-to-date picture of their practice and so sampled all new patients over the previous month (n=36).
• Figure 2 shows the simple data collection sheet developed to assess patient records.
• The criteria were defined in detail to make assessment as precise as possible
• A '1' was entered in the appropriate box when the criterion was met fully, a '0' inserted when the criterion was not met fully (eg. to receive a tick for the first criterion, a record should contain the patient's name in full - a record featuring only the second name would receive a '0').
• Simple percentages were used to sum up how well each criterion was met over all the sample and for each osteopath.

 

	Record no.
Criteria	1	2	3	4	5		35	36	Total	%
Personal Details
Name in full	1	1	1	0	1		1	1	33	92
Address including postcode	1	1	1	1	1		1	1	36	100
Telephone No:	1	1	1	1	1		1	1	36	100
Date of Birth	1	1	1	1	1		1	1	36	100
Gender	1	0	1	0	0		1	1	15	42
Occupation	1	1	1	1	1		1	1	36	100
GP name, surgery and tel. number	1	0	1	0	0		1	1	27	75
Presenting Complaint
Site of complaint	1	1	1	1	1		1	1	36	100
Date	1	1	1	1	1		1	1	36	100
Legible - mark 1 if assessed readable by practice manager	0	1	1	1	0		1	1	21	58

Analysis and discussion
The group of osteopaths each received a summary of their own record keeping along with anonymous copies of their colleagues' results. As a group they found that:

• they were meeting just over half of the criteria at near to or above the desired standard
• the remainder of the criteria were being met less than 60% of the time.

They were shocked by their findings and looked again at their professional guidelines:

• few had realised that signing and dating made notes more valuable as legal documents
• they initially disagreed about whether it was necessary to record gender if also recording a full name but finally agreed that names alone can mislead
• the group felt that they would be far more likely to record gender and the other patient details now that the reasoning behind their professional guidelines was clear to everybody.

Managing Change
The osteopaths:

• agreed to a re-audit of their notes in one month and in a further three months time.
• suggested printed case notes with prompts as a possible solution
• decided to wait for the results of the next audit round to see what effect increased awareness had on their record-keeping. Most standards were met on their first re-audit but they plan to re-audit again to see if this continues.

Defining the problem
Several complementary therapies involve the use of materials that need to be stored and treated with care. Acupuncturists need supplies of fresh needles and sharps boxes for used needles. Independent herbalists and homoeopaths will often need to store a wide range of preparations. Aromatherapists and massage therapists use a variety of different oils. Stores need to be secure and well-managed so as to avoid accidents and use of out-of-date materials. Storage conditions or maintenance can often be improved.

Benefits of this audit

• Use of a storage policy reduces fears about the safety of complementary medicines and techniques and can reduce waste
• Improved storage conditions should benefit patients as preparations will be more likely to retain their physical and chemical properties.

Purpose of Audit
This audit was done by two aromatherapists who worked from home. They both felt they remembered most of what they had been taught about essential oil storage during training, but both knew they could probably improve. They wanted to:

• read up on current thinking about storage
• check that their oils were being stored under ideal conditions
• perhaps write a policy on storage to remind themselves and patients who took oils home for their own use.

Criteria and standards
The aromatherapists wrote to their professional body for guidance and to one of the manufacturers of the oils. To save time they applied the following advice directly. Oils should be kept:

• in dark coloured bottles
• out of direct sunlight and preferably in the dark
• in a cool place, always below 15 degrees centigrade
• no longer than two years.

They decided that all of these criteria should be met 100% of the time.

Data collection

• The aromatherapists decided to assess their storage conditions with regular spot checks. They marked dates in their appointment diary at two-weekly intervals for the next three months.

• They used the form shown in Figure 4 to record their progress.

Analysis and discussion

• Within a month, the aromatherapists found a problem not covered by their initial criteria.

• They found they often made up too much of a specific essential oil and base oil mixture for use in a consultation. Rather than throw the mixes away, they stored them in their dispensing containers, hoping that they might be used on a return visit.

Managing change

• The aromatherapists decided to buy smaller dispensing containers to discourage themselves from mixing too much oil.

• They continued to make spot checks every two weeks for the next three months. Since they then consistently met their targets, the checks were reduced to one every four months.

Defining the problem
Complementary therapists often talk with patients about changes they should try to make in their day-to-day activities. Such advice could include giving up smoking or dietary or postural advice. Advice is likely to vary with each patient but it is sometimes possible to identify core information that should always be given to specific groups of patients. It is not always clear whether or how discussions about lifestyle should be recorded

Benefits of this audit

• a knowledge of research findings can help improve the advice you give to patients
• if lifestyle advice is recorded in a patient's notes then it is easier to follow-up.

Purpose of the audit
A chiropractor had found that she often could not remember what advice he had given to back pain patients on previous visits. This information often seemed to be missing from her notes. She wanted to:

• identify good practice in giving advice to patients presenting with low back pain
• describe the lifestyle advice currently given to patients and close some of the gaps between current and good practice.

Criteria and standards
The chiropractor got advice from a national association for people with back pain. Her college librarian found a selection of references to research papers and several literature reviews on postural advice. She discussed these papers with a friend in the college's research department. They agreed on some criteria for all back pain patients. These included the following:

• If patients drive, have a job that involves lifting or use a computer workstation, specific advice should be given and the discussion should be recorded in the notes (in 100% of cases)
• Advice given should be followed up at the next consultation (in at least 90% of cases - she decided that sometimes other discussions may have to take priority).

Managing change
The chiropractor started the audit by changing consultation sheets so that they prompted:

• discussion about activities that could affect back pain
• recording of key points of this discussion
• discussion of the advice at the next consultation.

Data collection

• Two months after introduction of the new consultation sheets, the chiropractor pulled all records for patients with low back pain
• She then also looked at the records for similar patients for the two months prior to the new sheets. 

Analysis and discussion
The chiropractor found that:

• before the audit, advice had been recorded in only 20% of cases.
• in the two months following introduction of the new sheets, advice had been recorded in 90% of cases, follow-up in 80% of cases.

Managing Change
Despite not reaching her target standards, the chiropractor was happy with the scale of the change achieved. She planned to continue auditing this aspect of care.

Defining the problem
A growing number of complementary therapists find that they need to write letters to their patients' General Practitioners. These might describe a course of treatment, or ask for information. Illegible, poorly structured correspondence is common in all health professions. It is possible to systematise letter writing to some extent so that it takes less time and still transfers vital information.

Benefits of this audit

• good communication between health professionals improves continuity of care, for example reducing the number of unnecessary tests patients receive

• a letter-writing system can reduce workloads for both writer and reader.

Purpose of Audit
A peer group of four chiropractors from several different practices used this audit because they were concerned about the relevance and content of their letters to GPs.

Criteria and standards
The chiropractors consulted their professional body's guidelines and met with local GPs. They decided that 100% of patients who consent to contact with their GP should have a letter written within a month of their initial presentation. These letters should:

• have no errors in legibility or spelling (in 100% of cases)

• contain a brief history, X-ray report, summary of diagnosis, treatment type, advice given, outcome and prognosis (in at least 90% of cases - the chiropractors thought these aspects were likely to be a problem and wanted to be realistic in early audit rounds)

• be short - no longer than one side of A5/ 250 words (in at least 80% of cases - the chiropractors felt this criterion was a lower priority).

Data collection

• The chiropractors drew up a data collection sheet (Figure 4)

• They pulled all of the last three months' records. These were searched for new patients giving consent for GP contact and letters written

• The last 20 letters for each chiropractor were examined by the clinic manager for compliance with the remaining criteria.

1. Number of new patients: 60
2. Marked as 'do not inform GP': 10
3. Number of 'eligible' letters [1-2]: 50
4. Number of 'eligible' letters sent within one month of presentation: 38
5. Proportion of 'eligible' letters that were not sent on time [(3-4)/3]: 24%
6. Sample of 20 consecutive letters sent since: 1/11/97.

Analysis and discussion
Each chiropractor received their results and those of the practice as a whole. They found that, for the group:

• only 76% of letters were being sent on time
• 90% were too long
• except for prognosis, the clinical content of letters was sufficient over 90% of the time • they did poorly on spelling and legibility (meeting standards in 50% or fewer cases).

Managing Change
The team was heartened by their findings on the clinical content but concerned by the late and over-long letters. They decided to:

• improve their clinic system for notifying chiropractors when letters were due
• try to shorten letters and use computer spellchecks
• do a second, identical assessment of letters in another three months, raising their standards for the criteria about clinical content to 100%.

Three months later the chiropractors found that all but one of the criteria were met - their record on recording prognosis was still slightly below target. They agreed to continue auditing letters to ensure that their standards did not slip.

Defining the problem
Patients who fail to complete a course of treatments may not get the full benefits of a therapy. There will be different reasons for not returning but misunderstandings about what treatment involves are common. Effective communication, both before and during a first consultation increases attendance at further treatments. Many therapists do not know how many of their patients fail to complete treatment or why this might be. It is often possible to improve communication.

Benefits of this audit

• It is essential to find out how much non attendance is a problem to a clinic before you take any steps to change clinic procedures. The perceived problem may not be too great after all.

• Feedback from patients is always enlightening, as long as good questions are asked. Patients who have not completed treatment are an ideal group to question about possible shortcomings.

Purpose of Audit
This audit was done by an osteopath who worked in a GP practice. He was concerned that several patients had recently not completed their planned courses of treatments. He suspected that they might not have known what to expect and had been disappointed. His audit aimed to:

• identify the numbers of patients discontinuing treatment
• find out patient perceptions about why they discontinue
• identify mismatches between patient expectations of treatment and reality and take steps to address these.

Criteria
The osteopath wanted to ensure that 100% of his patients:

• received diagnostic information at their first consultation; and
• felt that they had had treatment explained to them; and
• felt that they had been told about the expected number of treatments. 

Data collection - stage 1
The osteopath:

• wanted a longer-term picture so looked at all patients seen in the previous three months.
• identified patients who had failed to rebook appointments using the clinic's appointments register and patient notes.
• drew up a summary sheet that listed each patient who did not attend for the expected number of treatments.

Analysis and discussion - stage 1

• Out of 480 patients, 53 had not completed their expected number of treatments.

• The osteopath presented these findings to colleagues, including the practice manager and receptionist. The group decided that this was a high enough number for concern and talked about the kind of information patients might need.

• The GPs gave some feedback to the osteopath from their discussions with patients.

• The group drew up several questions to ask discontinuing patients. The questions asked about information requirements both before and during treatments.

Data collection - stage 2

• The osteopath drew up a draft questionnaire. This was tested on five discontinuing patients identified in stage 1. Figure 5 shows some questions from the final version.

• The questionnaire was mailed to the remaining discontinuing patients. See Appendix 1 for information about designing and mailing questionnaires.

Figure 5. Part of the questionnaire sent out to discontinuing patients

Analysis and Discussion - stage 2
When the questionnaires were back all the responses were typed onto a single, large summary sheet for quick reference. The osteopath asked himself:

• how do questionnaire responses compare with my criteria and standards?
• are there any surprises in my results? If so, what is surprising?
• does the questionnaire indicate particular problems with my service? If so, what can be done?

Feedback on the osteopath's skills in explaining treatment was encouraging but patients often reported that they did not feel that their condition had been explained. While the osteopath often remembered a discussion there was rarely mention of it in the notes. Patients reported practical reasons for discontinuing that were not related to treatment. 50% said that they would like better information before treatment.

The osteopath looked again at his criteria for good communication. Patients may forget about aspects of their treatment and it will often be better to record discussion of diagnosis in the notes and audit this. 100% patient satisfaction is extremely rare.

Managing Change

• The case note form was modified to prompt and record discussion about diagnosis and treatment. An audit of case notes is planned.

• A second questionnaire to a further 20 patients found that they still wanted more information.

• The clinic manager has drawn up a leaflet to give to patients on referral. Another questionnaire to discontinuing patients is planned to obtain feedback on the leaflet's contents. It is likely that lower targets for satisfaction will be set.

Defining the problem
Many therapists practice without systematically assessing their patients' progress. This can be a problem because we all have selective memories, tending to remember events that have gone particularly well or particularly badly. It is difficult to see things as a whole, both within the course of an individual's treatment and in the context for the totality of practice. Patients benefit if they feel that health professionals understand their experience of ill-health.

Purpose of audit
A practitioner of Traditional Chinese Medicine (TCM) used this audit to ensure he was looking critically at the progress of people attending an AIDS/HIV clinic. The practitioner also hoped to highlight areas where treatment may not have been working as well as hoped.

Criteria
The practitioner wanted to add completion of a short questionnaire to his usual consultation procedures. The questionnaire he chose, MYMOP2 (Measure Yourself Medical Outcome Profile 2) is one of a number of instruments developed recently to measure individual patient experiences of a course of treatment. MYMOP2 forms produce initial measures of:

• A primary and secondary symptom to provide the initial clinical focus
• An activity that is restricted by the symptoms
• The patient's overall feeling of well being
• The patient's attitude to medication use.

Follow-up MYMOP2 forms allow patients to continue scoring their initial symptoms and activity restrictions, to introduce new symptoms and to describe any changes in medication use. The scores translate into a visually effective graph for use within the consultation.

A recent study of an earlier version of MYMOP involving NHS general practitioners and complementary therapists (Paterson, 1996) showed that the questionnaire was practical to administer within a consultation. The practitioners reported increased awareness of patients' priorities.

The practitioner in this audit decided that he wanted:

• a focused discussion of treatment progress with all clients who continued with him beyond a third treatment
• to complete MYMOP2 forms for at least four treatments with all these clients if they were agreeable to this.

Data collection
Clients who had completed at least three sessions of treatment were identified from the clinic's register each week for a month and added to a list kept in the register. A set of four MYMOP2 forms was put in the notes to remind the practitioner to invite clients to fill one in at their next consultation. The completed forms were held in the notes and added to if necessary. At the end of two months the practitioner looked at the register list and the completed MYMOP forms.

Analysis, discussion and managing change
Eighteen of the TCM practitioner's clients saw him for three or more consultations. MYMOP2 had been used with all of these clients. All had given consent.

The TCM practitioners' patients reported that his use of MYMOP2 made them feel 'properly listened to'. He was able to present the graphs to colleagues and show that all but one of his patients' MYMOP2 main presenting symptom scores had increased, indicating improvement, or remained stable. He found that some symptoms were more commonly presented than others. These included fatigue, mental and emotional problems and ear, nose and throat disorders.

He reported that the collection of this data gave him increased confidence about his treatment approach but left him with more questions than answers. He did not plan changes to practice

.

Is this audit for me? This audit can improve communication between you, your patients and clinical colleagues. Other patient-generated forms may also be of value. Practitioners who want to use MYMOP2 should obtain a user's pack, complete with instructions (see Further help). This audit does not show whether a therapy is effective. It takes a procedure (MYMOP2 completion) that is thought to help and helps make sure it is used. The practitioner in this example found his audit did not identify areas needing change. It encouraged him to investigate published research in areas that did not seem to be responding to treatment.

National Centre for Clinical Audit
BMA House
Tavistock Square
London
WC1H 9JP

Tel: 0171 383 6451
Fax: 0171 383 6373
Email: ncca@ncca.org.uk
Web: http://www.ncca.org.uk

Provides guidance on audit methods and criteria for high quality clinical audit works in all sectors and with all professions set to become part of the National Centre for Clinical Excellence in April 1999

Eli Lilly National Clinical Audit Centre
Department of General Practice and Primary Health Care
University of Leicester
Leicester General Hospital
Gwendolen Road
Leicester LE5 4PW

Tel: 0116 258 4873
Fax: 0116 258 4982
email: gpaudit@le.ac.uk
Web: http://www.le.ac.uk/gpaudit/index.html

information service produces off-the-shelf audit protocols containing full instructions and evidence based review criteria publishes the journal Audit Trends can direct practitioners to their nearest Medical Audit Advisory Group (MAAG/PCAG)

Research Council for Complementary Medicine
27a Devonshire Street
London W1G 6PN

Tel: 020-7935-7499
Email:rccm@gn.apc.org
Web: http://www.gn.apc.org/rccm

Carries out, promotes and evaluates rigorous research in complementary medicine to encourage safe, effective practice and improved patient care holds in-house database containing over 80,000 references to complementary medicine research and development - database searches are carried out over the telephone

Primary Care Audit Groups/Medical Audit Advisory Groups
Nation-wide for local details, contact the National Institute for Clinical Excellence (NICE).

Complementary Medicine organisations
(for up to date list see professional bodies option under links button on main menu).

British Medical Association. Complementary Medicine: New Approaches to Good Practice. Oxford: Oxford University Press 1993. ISBN:0-19-286166-2

Department of Health. A First Class Service: quality in the new NHS. Department of Health 1998.

Department of Health. Clinical Audit: meeting and improving standards in healthcare. Department of Health 1994.

Fulder S. The handbook of complementary medicine. 3rd ed. Oxford: Oxford University Press 1996. ISBN: 0-19-262669-8

Fraser RC, Lakhani MK, Baker RH. Evidence-Based Audit in General Practice. Oxford: Butterworth Heinemann 1998. ISBN: 0-75-063104-X

Irvine D, Irvine S (eds). Making sense of audit. Oxford : Radcliffe Press 1991. ISBN: 1-870905-12-1.

Mills S, Peacock W. Professional organisation of complementary and alternative medicine in the United Kingdom, 1997: a report to the Department of Health. Centre for Complementary Health Studies, University of Exeter 1997. ISBN: 0-9531757-0-7.

Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. British Medical Journal 1996;312:1016-20. Users pack available: telephone 01823 282147.

Rowlands B. The Which? guide to complementary medicine. Which? Books 1997. ISBN: 0-85202-634-X.

Audit criteria - statements about the delivery of care that must be translated into measurements to assess the quality of care.

Clinical audit - a professionally-led initiative which seeks to improve the quality and outcome of patient care through clinicians examining their practices and results and modifying practice where indicated.

Clinical guidelines - systematically developed statements that assist in decision-making about appropriate healthcare for specific clinical conditions.

Clinical governance - a framework through which NHS organisations are accountable for continuously improving the quality of their services (see Department of Health, 1998).

Outcomes - the health, well-being or other state of the patient including the change in status attributable to previous care.

Medical Audit Advisory Groups/Primary Care Audit Advisory Groups - bodies set up within each NHS Health Authority to advise and encourage clinical audit and peer review.

This appendix starts with a step-by-step list of the action required in an audit. There then follows a set of responses to more technical questions about audit.

What needs to be done, when?
Experience suggests that effective audit requires each of the actions outlined in figure 6. Further details on these stages are available from NICE (National Institute for Clinicial Excellence)

Figure 6 Key stages in clinical audit
(adapted from 'Key points from audit literature related to criteria for clinical audit'. NCCA, 1997)

DESIGN Involve stakeholders. Involve the people who have a stake in the service in deciding on topics for audit and why audits should be done. Select important topics. Decide what to audit through a systematic process. State objectives. State why the audit is being carried out. Use explicit measures as a basis for collecting data. Reflect good practice. Use measures which reflect the best available evidence of good practice. Ensure that the measures used are acceptable to the clinicians whose care will be assessed. Use measure of processes known to be associated with outcomes in preference to outcomes measures alone. Define audit case selection. Describe the cases, episodes, events, situations, or circumstances for the audit. MEASURE Pilot data collection procedures and forms to ensure the reliability of data. Respect ethics and confidentiality. Ensure that all aspects of the data collection process are consistent with accepted ethical and confidentiality principles. Analyse audit data. Use appropriate methods to group and analyse audit data. EVALUATE Present audit data. Identify shortcomings and their causes. For cases in which variation from the audit measures can not be explained or clinically justified, identify shortcomings in practice and their potential causes. Name the specific improvements in practice which are needed. ACT Devise an action plan. Implement action. Carry out the action plan. REPEAT FOR IMPROVEMENT Repeat the process. Reaudit as frequently as required and as quickly as possible until needed improvements are achieved and sustained.

How do I decide which topic to choose?

Try applying the following criteria. Is the topic:

• Likely to benefit patients?
• Likely to benefit my practice/clinic?
• Relevant to my professional development?
• Relevant to the development of the profession as a whole?
• Significant or serious in terms of the process and outcome of client/patient care?
• One where there is potential for improvement?
• Capable of holding the interest and involvement of team members?
• Likely to repay the investment of time, money and effort involved?

How do I do sample cases?

• Get advice - eg. from your local Medical Audit Advisory Group.
• Check that you need to sample. In general, the greater the number of cases you are interested in, be they sets of notes, patients, events, the more helpful it is to sample.
• Samples do not need to be very large. You are aiming for a 'good enough' estimate of practice, not a precise one.
• You may, however, want a relatively complete picture, eg. one that captures possible day-to-day variations.
• You can sample using random numbers taken from a table or a computer. Before this you need to give a unique number to each one of the cases you want to sample from.
• A simpler method, called systematic sampling is often suitable. You need to know the number of cases of interest and need them to be in some form of order. Choose the first case at random and then select the proportion you want to sample (say every tenth one). For example, to select 20 records from 2000 of interest you might select the 45th at random, then take the 145th, the 245th and so on, in turn.

How can I use questionnaires?

• Get advice early on.

• Don't reinvent the wheel. Look first to see if a suitable questionnaire already exists. This is especially important if you are asking patients about complex areas of care such as their presenting condition, symptoms and current or past state of health.

• Questionnaires should be piloted first on a small number of patients. This is to make sure that the questionnaire is quick to complete and that questions and directions are clear and unambiguous.

• A personalised covering letter on letterhead paper is essential. This should explain the reason for the questionnaire, the value of each response and provide contact details for any questions.

• To encourage frank responses, if possible, arrange for the questionnaire to be addressed to the clinic manager or some other neutral source.

• Similarly, stress that all responses will be treated confidentially. Get help in analysing responses so that you cannot link feedback about your service to an individual that you may treat again. Anonymity should be used only when a topic is very sensitive or you cannot set up a system to treat responses in confidence. Anonymous questionnaires result in low response rates.

• Identify questionnaires with a unique code so that reminders can be sent. This is vital to get a good response rate. A poor response rate usually means that your findings over-represent the opinions of people with strong views.

• A stamped- addressed envelope improves response rate, as does a short, clearly designed questionnaire. Similarly, put easy questions first and lead up to more difficult ones.

APPENDIX 2 - GUIDANCE FOR INDIVIDUAL PRACTITIONERS

There has been confusion in the past as to what clinical audit can do and what it requires. The following points may help avoid misunderstanding and ensure that audit is an effective approach for complementary therapists and patients.

Can clinical audit help me find out if I am effective?

• One of the golden rules of audit is that it should address things that are under the control of clinicians. This is why most successful audits look at processes or structures that are known to result in effective care. The outcomes of patients, complementary or otherwise, are influenced by many factors. To find out which factors are important, we need research, not audit.

• Outcomes monitoring is often confused with audit. Monitoring of outcomes can help inform research. It is unlikely to lead to audit's main goal: improved care in the short term.

• Choose clinical audit if you want something that can have direct, positive effects on patient care within a short timescale.

Do I have the skills and technology?

• Clinical audit requires change management and people skills far more than it does technical skills. It is rare, for example, for measurement in audit to involve anything other than simple percentages and totals.

• Don't equate audit with Information Technology. The simplest audits can be done with paper, pens and a calculator. Dealing with unfamiliar computers or new software could take up all your audit time. If you need access to data that is held on computers build extra time and support into your project to deal with this. Many audit topics can be addressed using simple paper records or questionnaires. Likewise, a helpful librarian is usually worth more than access to the Internet.

Who can help?

• National and local audit bodies can give support and advice (see Further help).

• Get in touch with like-minded practitioners. In the absence of national guidelines on your chosen topic you can set your own. If you can reach a consensus with a group of peers you are likely to have a fuller picture of the different aspects of the topic you are auditing. This group will also be a source of ideas on how to bring about change.

• Put pressure on your therapy organisation to support the audit work you and your colleagues do. Individual practitioner commitment to patient care deserves concrete professional support. Support could take many forms, including credits for audit as part of continuing professional development, improved library or information services. Several professional bodies are developing audit programmes (see Further help).

• Work with your professional body to help them provide appropriate guidelines for good practice.

• Involve colleagues in your audit work. If someone may be affected by changes in the area you are auditing, involve them - before the standard setting stage if possible. Do not overlook practice managers and clerical staff. Their co-operation and perspectives are essential.

APPENDIX 3 - SUPPORTING AUDIT IN COMPLEMENTARY MEDICINE

In doing audit, complementary therapists may often need to deal with practical difficulties which other health professionals may not face. This section aims first to identify these difficulties and highlight areas likely to need extra support. It then makes recommendations for professional groups that want to promote audit as a routine activity among members.

Assisting individual therapists do audit
Many complementary therapists will have trained at private institutions with only weak or recently made links to the UK's academic system. While there are notable exceptions:

• Research is only just beginning to appear on the complementary therapy curriculum. Many complementary therapists will be under the impression that audit can be used to investigate whether or not an intervention works.

• Access to academic or peer support and to useful facilities is often poor. Unfamiliarity with the extent of library facilities can lead to delays at the critical early stages of audit.

Again, the degree of development of professional bodies varies considerably for different therapies, but:

• Incentives for training or professional development post qualification are often lacking. Schemes for continuous professional development are restricted to the most established professions. Individual practitioners may not always get appropriate professional recognition for audit activity.

• For all but a few professions, the development of clinical guidelines or protocols is at an early stage. Guidance on good practice is urgently needed from the less developed therapy bodies.

• There is very little controlled research in complementary therapies to guide clinical decision making. The complementary medicine research- base provides some evidence that certain interventions may be of use for some patients. It will rarely support a specific intervention in a given clinical situation. Audit criteria are unlikely to be informed by controlled complementary medicine research in the near future.

Individual complementary therapists are therefore often likely to need:

• well-timed advice on the scope of resources available for audit.

• clear advice at an early stage on the strengths and limitations of the audit approach.

• support so as to develop their own criteria for good practice.

Promoting audit within a profession

• Much can be learned from other professions. For example, the work of community pharmacists has much common with complementary practice. Community pharmacists are sole-practitioners. They can feel isolated from their peers and have minimal access to funding for audit. The chiropractic and osteopathic professions have also been promoting audit for some time.

• Recognise that practitioners need motivation and support to do audit. A first stage would be to raise awareness of the benefits of audit. To get a significant number of practitioners actually auditing their work will require further commitment from the profession as a whole. Motivation can come from many sources, including altruism, learning opportunities but also business benefits, increased recognition and reduced problems. Support will need to include access to expert individuals and written materials.

• After raising awareness, increase acceptance. As is the case for any change in practice, the uptake of audit will be slow unless attempts are made to reduce practitioner concerns (see Figure 7).

• Link audit to other professional initiatives. Audit has an obvious place in education. Simple audits are easily done by trainee therapists. They can be a good way of providing an introduction to systematic enquiry that has immediate benefits for patients. Nationally co-ordinated audit projects can help practitioners look at how their own practice compares with national standards and with the work of their peers.

• Respect individual needs, expertise and ability. Audit can be perceived as threatening. Feedback about performance should always be confidential. Those who will be affected by changes should be invited to participate early on. Change needs to be appropriate to individual settings. Practitioners and their patients are a unique source of information about care.

Common concerns about audit - and some responses

• "Audit is about being assessed 'from above'". - audit can be self-directed by individual practitioners or co-ordinated by a professional body but it should always be confidential.

• "It's not me - it's the others!" - but audit raises standards throughout a profession.

• "We already do it" - most practitioners do look subjectively at their own work, but this can be misleading. Audit provides a more objective basis on which to make decisions.

• "What's in it for me?" - this often goes unsaid. It is important to acknowledge motivation.

• "Not enough time"/"I'm not being paid for it" - these are both questions of priority. Other professions report that audit has repaid the effort spent.

• "I don't want to standardise"/"What I do is an art" - but all artists use techniques.

• "Outcomes research is more important" - outcomes research helps with audit but is not a prerequisite. We need audit to improve the quality of what we do. Outcomes research helps us find out what we should be doing.

© RCCM February 1999.