How do we know this works?

“You can’t use an old map to explore a new world.” Albert Einstein

Why do we need evidence?:

Whilst the majority of colleagues intuitively understand LfE, a minority of colleagues reserve judgement and request to see “hard evidence” to support its implementation.

“…show me the interventions for all this touchy-feely stuff; I need hard facts and tools”  NHS senior leader (anonymous)

Initially, I ignored these comments as the benefits of LfE were obvious to me, and the rapid uptake of the initiative was evidence that it was perceived as useful by many staff in the NHS.  However, over time, I became curious and set about looking for evidence to support LfE. 

This blog post is a summary of the evidence I presented at the #LfEconference18 in November 2018 at Birmingham Town Hall.  This is by no means an exhaustive presentation and I hope that, over time, our Community of Practice will collectively build on the evidence presented here.  The library section on this website is updated regularly; please contact me if you wish to suggest articles / data / other evidence for our site.

How to articulate the message:

Whilst researching the evidence base for LfE, I also started thinking about how best to articulate the message to those who don’t intuitively understand it.  It seems that we do not all speak a common language.

A recently published report from the Carnegie Institute describes two different languages (or “lexicons”) used by policy makers: the rational lexicon and the relational lexicon.  This report (written by Julia Unwin) is primarily about kindness in public policy.  Whilst not directly about healthcare, the content of the report was useful in highlighting different ways to articulate the potential benefits of LfE to policy-makers and leaders in healthcare. 

Occasionally I hear about a colleague who wishes to implement LfE in their organisation, but their manager puts a block on the implementation due to a lack of evidence or “rational” metrics (see the quote above).  According to the Carnegie report, the manager in this example is fluent in the “rational lexicon”, but may have difficulty listening to or understanding a rationale presented in the relational lexicon.  In the words of Julia Unwin, framing arguments in the relational lexicon (e.g. about kindness) may be perceived as “unhelpfully interrupting the adult flow of conversation”.

Taken from the Carnegie report, here are the characteristics of the rational and relational lexicons:

Both lexicons have their risks and benefits if they are used exclusively, and I believe it is possible to recognise both types of language when communicating about LfE (or any other initiative or activity).  In reality the distinction between the two lexicons is not absolute, and it shouldn’t be taken as a fixed dichotomy.  Nevertheless, I have found the model very useful when discussing the potential benefits of LfE and the evidence to support it.

To the evidence:

The evidence collected in this blog post is divided into the following sections:

1. Stories / anecdotes
2. Staff survey data
3. Data from the PRAISe study
4. Related science and research from outside the LfE movement

Stories and anecdotes:

Although anecdotal evidence does not stand up well to academic critique, story-telling can be a very compelling way to articulate a message.  Each LfE report is a short story: a story describing an episode of excellence.  Simply reading a report, and discussing it with the staff members involved, is often enough to demonstrate worth of the initiative.  The excellence described in the reports is typically a type of non-technical skill, and recipients of reports frequently report “going about their work differently” as a result of receiving recognition.  Often, they were not aware that there actions had had such a positive impact until they received the report.

Within the Community of Practice, there is a growing collection of stories of impact of the initiative, within various different healthcare settings. One example is captured in this excellent podcast describing the implementation of LfE in Derby.

We have also uploaded a few videos of staff describing their personal stories and insights related to LfE on our youtube channel.  Stories describing impact of LfE are often shared informally, and we would like to start compiling some on our library page of the website.  Please contact us if you wish to share stories of impact of LfE.

Staff surveys:

Many staff surveys have conducted across the LfE Community of Practice, some of which have been displayed at our conferences (see our conference poster compilation).   At BCH, we have conducted two large surveys of our PICU workforce.  These surveys were conducted in 2015 (1 year after implementation) and 2018.

The BCH staff surveys show highly positive perceptions of the value and impact of LfE within our department.  The 2018 survey is still under analysis, and is being replicated in other centres currently, but here is a summary of a selection of questions from the surveys along with the proportion of positive responses:

Year of survey	Question	% of positive responses
2015	By reporting excellence, I am helping to improve patient care	86%
2015	I learn best from studying examples of good practice	87%
2015	Excellence reporting can improve team morale	93%
2015	Excellence reporting can boost my motivation	87%
2018	Receiving excellence reports increases the likelihood of me practising in a similar way in the future	89%
2018	Excellence reporting leads to improvements in the safetyof care we provide	85%

We will be carrying out further analysis of the 2018 survey, and to replicate it in other centres over the next few months.  Please contact us if you wish to use the questionnaire.

PRAISe project data:

The PRAISe project was designed as a proof of concept project to measure the impact of LfE interventions (positive reporting and appreciative inquiry) on an area of clinical interest.  We chose antibiotic stewardship / sepsis as a test area due to a connection with my clinical interests, but the PRAISe methodology could be applied to almost any Quality Improvement situation. 

The project is summarised in full in the Health Foundation report, and we are currently writing a manuscript for submission to a peer-review journal, but some of the results are shared below.

We found that positive reinforcement via excellence reporting and appreciative inquiry interviews was associated with an increase in quality in some of our measured processes (all of which were orientated around behaviours of healthcare professionals). There seemed to be a dose-response effect, in the sense that the largest improvements were seen in the processes receiving the highest concentration of positive feedback. 

The best example was seen in improvements in antibiotic prescribing: the rate of gold standard prescriptions increased during (and after) a period of positive feedback (the red arrows show the start and end of the intervention period):

Our summary interpretation of the PRAISe project is that “if you show staff what they are doing well, they will do more of it”.

Related science and research from outside the LfE movement.

Theoretical evidence from neuroscience:

“Failure is the best teacher”

Some scepticism about LfE emerges from the assumption that we learn best from failure, and therefore we should concentrate our efforts on recognising and interrogating episodes of failure (e.g. error and harm).  This is compelling to anyone who has ever made an error (i.e. all of us).  However, it ignores the fact that success is actually highly instructive – in fact, it is probably more instructive than failure.

Experimental evidence from neuroscience shows that success leads to faster and stronger memory formation (i.e. learning) than failure.  However, this learning can only take place if feedback has been recevied – i.e. we need to know if we have achieved success or failure.  Without feedback we are not aware of the outcome.  The outcome of many of our interactions in healthcare is not always made known to us – this is especialy true for interactions relying on non-technical skills.

This is where LfE reporting has a unique advantage: very commonly, recipients of LfE reports state that they were unaware of the positive impact of their intervention or behaviour. 

LfE therefore provides the feedback to allow learning from success – in an environment where the prevailing approach to learning is to highlight failure.

Staff engagement:

One of the key functions of LfE is to provide recognition to staff (individuals or teams) who have performed excellently.  We know from the thousands of reports we have received, that many of these episodes of excellence are everyday activities – e.g. lending a hand; supporting a peer; going above and beyond;  showing kindness to a patient or colleague…

Thus, the LfE report serves as a means of recognising a colleague’s excellent work.  Recognition is a key factor in staff engagement: staff who feel recognised are more engaged in their work.  This has been demonstrated in several large studies; a compelling example comes from the WorkTrends (TM) survey from IBM:  – a survey of >19,000 workers in 26 countries, from a cross-section of industries.

The survey results demonstrate the positive relationship between recognition (measured as  agreement with the statement, “I receive recognition when I do a good job”) and staff engagement.

The importance of staff recognition is also shown very clearly in this report from Gallup:

Employee Recognition: Low Cost, High Impact.

But, does staff engagement correlate to any meaningful (and measurable) outcome?  In short, yes.  West and Dawson’s 2012 report for the King’s Fund (Employee Engagement and NHS Performance) clearly demonstrates the positive relationship between staff engagement in the NHS, and multiple  important outcomes including quality of services and quality of financial performance:

Thus, LfE provides recognition for staff members; staff recognition is related to staff engagement, and staff engagement is related to multiple important, measurable outcomes.

In conclusion:

I have described evidence to support LfE from within the initiative, and from sources outside the LfE movement.  But, this is not an exhaustive report:  I have scratched the surface, and there is a lot more to discover and explore.  I have also not looked at the potential risks of LfE.  There may be a shadow side of which I am not aware.  Whilst I continue to look for evidence and data from this initiative I welcome comments below and via our forums.

 

Adrian