Outcomes FAQ

• Where can I find data on my surgeon?
• The data is being published in England. But I live in Wales / Scotland / Northern Ireland. How does this help me?
• Where does the information come from?
• How do I know it's reliable?
• What has the data been used for up until now?
• Is all surgery covered in these specialties?
• Why are the outcomes not presented in a standard format?
• Is the data complete?

Where can I find data on my surgeon?

You can view the outcomes data of your surgeon from the national clinical audits via the My NHS (NHS Choices) and via the individual surgical specialty associations or their audit websites. The thirteen surgical areas are: adult cardiac surgery, bariatric surgery, colorectal surgery, head and neck surgery, interventional cardiology, orthopaedic surgery, thyroid and endocrine surgery, upper gastro-intestinal surgery, urological surgery, vascular surgery, lung cancer, neurosurgery and urogynecology.

Your surgeon’s name, General Medical Council (GMC) registration number and place of work should be listed, with the data.

The data is being published in England. But I live in Wales/Scotland/Northern Ireland. How does this help me?

Publication of surgeon-level data is, at present, a requirement of NHS England. Many of the specialty associations are UK-wide and may decide to make data available on surgeons working outside of England, information which will be available on specialty association websites. It will be up to the Governments in the devolved nations to decide whether publication should be mandated.

Where does the information come from?

Each of the surgical specialty associations carry out clinical audits of the work of their member surgeons and of procedures carried out in the UK. These collections of information are sometimes called registers.

A number of audits and registers are well established. For example, the UK National Adult Cardiac Surgery Audit has been running since 2005, and the orthopaedic audit, known as the National Joint Registry, has been running for eleven years. Others, for example, the audit of bariatric surgery (for obesity), are newer.

How do I know it is reliable?

The data has been validated and analysed by the surgical specialty associations to work out the appropriate standard for a surgeon, and to define acceptable and unacceptable variation from this standard.

Those surgeons that are far from reaching this standard, or are doing much better than it, are described as outliers (ie they are out of the ordinary from the mean average).

What has the data been used for up until now?

The data has been scrutinised by the specialty associations to pick up any potential areas of poor performance. Where necessary, there has been further investigation by the surgeon's employing Trust.

Is all surgery covered in these specialties?

No. Most surgical specialties cover a wide range of operations, and at this stage the published data only covers a small number of these procedures. The selection process takes into account factors such as: what procedures are currently covered by audit/registry data; how frequently the procedure is undertaken; and whether it is considered a good indicator of skill. For example, orthopaedics have chosen to look at hip and knee replacement surgery.

As a result your surgeon’s data may not be available simply because they do not carry out procedures for which this data is available.

Why are the outcomes not presented in a standard format?

The format of the data is consistent on the My NHS website. On the Specialty Associations websites, each association has presented the data in the way they feel best fits the procedure as the procedure and data collected have been presented in a way that most accurately portrays the specialty. Some associations will be using funnel plots, a widely used method to graphically collate audit data and show outliers.

For more details, go to the specialty associations’ websites.

Is the data complete?

A large amount of data has been gathered, but it is not entirely complete. In any healthcare organisation, data can go missing or be incomplete. Also, for a number of audits, surgeons have been required to submit their own data, with little support at hospital level to enable them to do this.

Surgeons have been asked to consent to the release of their information. All surgeons requested to submit their data did so. The data set includes over 5,000 consultant surgeons covering 28 procedures.

• What does the information tell me?
• My surgeon has a great reputation locally, but s/he is an outlier - does that mean s/he is a poor performer?
• What is risk-adjustment?
• How is the adjustment done?
• If the data is adjusted to take into account the mix of patients and the difficulty of the surgery, surely that means the outliers are poor performers?
• Why are there differences in performance between the very best surgeon and the poorest, even though they're all within accepted limits?
• My surgeon is near the middle line. Does that mean nothing can go wrong?
• Why isn't the data presented as league tables? Patients want to know who is best.

What does the information tell me?

First, it shows the number of times a given procedure has been performed by a surgeon over a year, so you can see if your surgeon carries out the surgery regularly or rarely. Second, you can see how close they are to the average standard. If they are doing much better or worse than the average, they are described as outliers. An outlier can be identified on the My NHS website with a grey exclamation mark next to the consultant’s name.

Each audit will be providing information on mortality rates but some will also be providing information on other clinical measures, such as readmission to hospital or length of stay in hospital.

My surgeon has a great reputation locally, but s/he is an outlier - does that mean s/he is a poor performer?

Not necessarily. Although s/he may be further from the average than most, s/he still may be within the range considered acceptable by the surgical specialty associations. Even if s/he is an extreme outlier, there may be complex reasons why the risk-adjustment has not been effective, or there may be individual circumstances explaining the difference. To be confirmed as a genuine outlier will necessitate further investigation by the employing Trust concerned with assistance from professional bodies if required.

What is risk-adjustment?

Patients vary by age, sex and the number of other illnesses they have – known as co-morbidities. Patients with conditions such as diabetes or respiratory problems are higher risk because of their complex health needs. Some surgeons may have many complex patients, while others may have far fewer. This is known as the patient case mix.

This needs to be taken into account when considering figures such as mortality rates, as higher risk patients are more likely to die. Also, some procedures are inherently riskier than others. This needs to be taken into account too.

How is the adjustment done?

The data is adjusted by statistical experts using complex methods. In the case of mortality, the charts show what the rate would have been had each surgeon operated on the average case mix.

If the data is adjusted to take into account the mix of patients and the difficulty of the surgery, surely that means the outliers are poor performers?

Again, not necessarily. Sometimes risk adjustment cannot tell the whole story. Some surgeons are taking on extremely difficult and challenging surgery on very sick people, so have relatively high death rates. Others may be developing new techniques. They may be shown as outliers, but they may also be the leading specialists in their field.

Risk-adjustment aims to reduce these factors so that surgeons are not inaccurately represented. However, this risk-adjustment process is new and as a result may require improvements over time.

Why are there differences in performance between the very best surgeon and the poorest, even though they’re all within accepted limits?

As in all areas of work, some people are more skilled than others. In surgery, this may be to do with the training, their years of experience or the range of operations they perform. What is important is that patients and the NHS can know whether a surgeon is within an acceptable standard, based on the best professional advice.

It is also important to remember that there are many reasons why this data may show differing performances between surgeons, and for some specialties in particular, may not be a reflection of surgeon skill at all. It is important to discuss any concerns you have with your GP or surgeon.

My surgeon is near the middle line. Does that mean nothing can go wrong?

All surgery carries with it a certain level of risk. The surgeon is part of a team of people contributing to your care. That is why it is important to also look at data available for the unit in which the surgical team carry out a type of surgery (unit data).

If you have any concerns whatsoever about the risks of a surgical procedure or your surgeon’s data, speak to your surgeon beforehand.

Why isn’t the data presented as league tables? Patients want to know who is the best.

It is important not to misinterpret the statistics. League tables can be crude and misleading in this area because surgeons all carry out a different range of procedures, even within the same specialty, and carry out a different number of cases each year, on both high and low-risk patients. Surgeons who appear to be significant over-performers may be nothing of the sort or the differences may be small. If data are misinterpreted, surgeons may in future refuse to operate on high-risk patients who could benefit from an operation if they think they are going to be ranked or placed in a league table. They may become too conservative in their treatment – known as risk-averse – and lose the spirit of medical innovation.

• What should I do if I am concerned that my surgeon is an outlier?
• If being an outlier doesn't necessarily mean the surgeon is a poor performer, how can the data be used in a productive, useful way?
• How can I use this information to choose the best surgeon for me?
• But I have been given a surgeon who has done far fewer cases than another. I don't want a beginner. Can I ask for a different one?
• What is more important to consider: the team's results or the individual surgeon's?

What should I do if I am concerned that my surgeon is an outlier?

Don’t panic or assume the worst if your surgeon is an outlier. Talk to your GP, your specialist nurse or your surgeon who will be able to explore this further with you.

If being an outlier doesn’t necessarily mean the surgeon is a poor performer, how can the data be used in a productive, useful way?

The publication of these data represents a starting point for the medical profession, the Government and regulators. There is strong evidence to suggest that medical practice improves as a result of publication of data like this 1. Surgeons are more likely to concentrate on their own performance and data, and seek assistance from colleagues to get things right. It will encourage them to really consider whether they are operating on the right patients and doing the right operation at the right time.

How can I use this information to choose the best surgeon for me?

Look and see how many operations he or she has performed to understand their familiarity with the procedure. Also look at the unit data as surgeons work within teams, and they all play their part in its success.

But I have been given a surgeon who has done far fewer cases than another. I don’t want a beginner. Can I ask for a different one?

Surgeons mostly work in teams. While the individual surgeon may not have carried out a high volume of procedures, less experienced surgeons work under close supervision, and all operations are carried out by a team who are likely to be highly experienced. If you are concerned, you should speak to your GP or surgeon.

What is more important to consider: the team’s results or the individual surgeon's?

Modern medical practice is increasingly focused on working across disciplines and aiming for better care by specialists working together. Rather than focusing on your individual surgeon, the team s/he works within is probably more significant.

• My surgeon is not included - does that mean he or she has something to hide?
• My surgeon is not a consultant. Can I find out information about him/her?
• Why aren't all the specialties publishing data at the same time?
• Why aren't all operations and procedures covered within the thirteen specialties?
• How can I easily compare my surgeon with others in the same unit when I don't know their names?
• How can I compare my hospital unit with others around the country that are of a similar type?
• My local unit seem to be an outlier, but the Trust managers say their own data proves them to be good. Who do I believe?
• What will happen to surgeons found to be significantly underperforming? Will action be taken by the College / Department of Health / General Medical Council? Who will investigate?
• What about operations in private hospitals? Are they included in the audit?

My surgeon is not included - does that mean he or she has something to hide?

More than 2,500 consultant surgeons are not included because they do not perform any of the procedures which have been chosen as a priority for publication at this stage. All surgeons who were requested to submit data have done so.

My surgeon is not a consultant. Can I find out information about him/her?

No, this data covers only consultants at this stage. These audits will also allow you to find out about the unit data.

Why aren’t all the specialties publishing data at the same time?

Some of the audits and registries are relatively new, and it has been harder for them to accumulate data in the form needed.

Neurosurgery specific outcomes data will be published in December while Urogynecology will be released in March 2015.

Why aren’t all operations and procedures covered within the thirteen specialties?

This is the beginning of a process. Operations that define a surgeon’s ability and are done frequently enough to get statistically meaningful data have been chosen, but this has been tougher for some specialties than for others.

How can I easily compare my surgeon with others in the same unit when I don't know their names?

At this stage the data lends itself to informing patients about the practice of their consultant against the backdrop of the national average, not necessarily to comparing all consultants from the same unit. However, patients will  be able to see which unit(s) each consultant works at.

How can I compare my hospital unit with others around the country that are of a similar type?

This initiative has been to produce consultant level information so the focus has not been on unit level. Some audits (eg cardiac surgery) will show units against the backdrop of other units. For others this additional level of information will follow later.

My local unit seems to be an outlier, but the Trust managers say their own data proves them to be good. Who do I believe?

Individual Trusts have responsibility for the quality of care being provided and will need to assure themselves it is of a high standard. The Trust will work with the audit provider to look at issues of data quality and completeness. If there are problems with this, we would expect the Trust to investigate further. Commissioners will also want to see evidence that perceived poor performance has been addressed appropriately. The College offers an invited review service to assist Trusts and commissioners in deciding whether there is a cause for concern about an individual surgeon or surgical unit.

What will happen to surgeons found to be significantly underperforming? Will action be taken by the College / Department of Health / General Medical Council? Who will investigate?

Individual Trusts have responsibility for ensuring the quality and safety of the surgical services they provide. They will need to investigate perceived poor performance and take appropriate steps to remedy the situation if there is a cause for concern.

The College offers an invited review service to assist Trusts and commissioners in deciding whether there is cause for concern about an individual surgeon or surgical unit.

Patients should be assured that the specialty associations all have processes in place to identify, and, working with Trusts, to investigate surgeons and units that are identified through audit as performing outside of acceptable limits.

In addition, revalidation is a new process of regulation for all doctors that practice medicine in the UK. Every five years doctors will need to show that they are still fit to practise. The outcomes data for specific surgeons will be a subject of discussion at these meetings where fitness to practise is being regularly assessed.

What about operations in private hospitals? Are they included in the audit?

Where NHS patients are operated on outside NHS facilities, these procedures should be reflected in the outcomes analysis. Where patients fund their own treatment, this may not be reflected. Individual associations will provide further detail on this.