If only

In my analysis of the Improving Access to Psychological Therapies (IAPT) programme’s recent data review, I was strict in my interpretation of the figures, and the final 4% success rate was a dismal result. It does no harm, and it might even do some good, to dream of what might have been.

Note: To understand what follows, you will probably need to have read my previous post, Sums, which describes the data analysis in detail and contains links to the source documents. In order to cut a long story short, I only give a brief outline here.

Still keeping strictly to the published figures, the site-specific tables provide a way to dream of a better IAPT in the future. The 34 sites varied, sometimes widely, in the results they provided. What if all the sites were to adopt the very best practices?

Total referrals

For example, the average number of patients referred to a site in the year was 4,290. I used this to estimate the total number of referrals to all 34 sites as 145,865.

But Site 31 received 13,354 referrals (SS1 ‘Total records’), much larger than the average. What if all 34 sites had dealt with this number of referrals? More than three times the number of patients, 454,036, would have been dealt with in all.

Best practice

I used this technique to determine the best practice results at each step. Instead of explaining every step in a separate diagram as before, here they all are in a single table. The ‘Excluded’ column matches the descriptions in the ten diagrams of my previous analysis.

The ‘At best’ column tells you what the number excluded would have been if all the sites had been as good as the best site at that step in the calculations. The ‘Included’ column is the number still included after that step.

Note that the ‘Best site’ column usually contains a percentage for the number excluded, but I sometimes had to calculate this from another number given in the site-specific data. For the three percentages marked with an asterisk (*), I copied date into a spreadsheet to calculate percentages for all the sites, and then identified the best site. SS25 and SS25 data is only for the MTR1 recovery criterion, as before.

Excluded	At best	Included	Best site
No data	0	454,036	Almost all sites
No initial assessment	0	…	Site 32: zero (SS1)
Still in the system	126,540	327,496	Site 27: 27.9%* (SS1/21)
Only one contact	0	…	Site 32: zero (SS3)
Presumed no treatment	0	…	Assumed
No final assessment	0	…	Site 33: zero* (SS24/25)
Not ill	27,291	300,205	Site 18: 8.33%* (SS24)
Treatment not complete	46,832	253,373	Site 24: 15.6% (SS21)
Treatment failed	107,684	145,689	Site 4: 42.5% (SS24)
Relapsed	1,665	144,024	Site 12: 6.1% (SS24)

So the net percentage recovered in this imaginary scenario is:

32%*

*144,024 as a percentage of 454,036

That’s eight times better than the actual performance of 4%, and more than twenty-five times the number of recovered patients. According to the available data, these results could be achieved simply by propagating best practice at every step across all the IAPT sites.

This time I have almost certainly gone too far to paint a rosy picture of what could be achieved. For example, perhaps Site 32 was able to assess everyone because it had only a small number of referrals in all. Or perhaps not — there’s nothing obviously difficult about scaling up from a small number of assessments to a large number.

Picture it

For an easier comparison, here’s a simplified and approximate picture of the actual performance (using the data from Sums) for 25 people referred. One of the 25 (4%) recovers. You can see that the biggest problem is the number not assessed at all:

And here’s a similarly simplified and approximate picture of the ‘best practice’ performance for 25 people referred. You can see that many more people receive treatment, and now the biggest problem is the number of people who have treatment that doesn’t work:

But getting any treatment at all for all those people was just a dream, remember. Although the whole point of the Improving Access to Psychological Therapies programme was to improve access, it turns out that access is still a problem. And although some sites seem to have solved that, it turns out the access problem is masking another problem — the quality of the therapy being delivered.

A private psychotherapy practice would not be able to stay in business with those numbers of failures. Imagine a hairdressing salon, motor vehicle repair shop, or refuse collection service with results like that. These figures for the first year of IAPT leave me still saying, “If only…”

Advertisements