The ASA received complaints about claims being made by a driving instructor on his website. Kelvin White, of Tiverton, was claiming a first time pass rate of 80% and someone made a complaint.
He has since removed the claims. In reading his comments in that article in This Is The West Country, he seems to be another instructor who either doesn’t understand statistics, or one who understands them all too well. You see, the problem is that ADIs are always looking for a USP (unique selling point) and although pass rates are hardly unique, very high ones are perceived as being so.
No ADI in the UK with a statistically significant number of tests under his belt can claim 100%. However, if he is selective with his data then he can sometimes get very close.
At one point this year (2013), my first time pass rate was 100%. But at the start of last year even my overall pass rate was 0% (a single pupil failed his test four or five times in a relatively short period, and I don’t think anyone else had a test in the same period). Pass rates are great when they’re high, but they are awful when they’re low, and people who put them on their websites are not going to report low ones.
I don’t quote pass rates on my school website. I mention them on here from time to time, and I only use figures for the calendar year in question because that’s all that matters to me. It means a warts-and-all figure – it’s 57% for the year to date, with 12 passes from 21 tests – which is the only figure that comes even close to meaning anything. But if I wanted to be creative (and deliberately misleading), I could quote the percentage as a function of the number of pupils who have had tests, and that puts my percentage up to 70%. And although it didn’t occur to me to check it again until now, my first time pass rate – expressed as a percentage of my test passes – is 67%, although as a percentage of all tests it is only 38%. And I could really make the figures look good if I took out certain pupils – who would know?. You can see the problem here.
And claims made by individual schools for marketing purposes completely overlook the differences in pass rates in different parts of the same county and especially across the country as a whole. Test centres out in the sticks usually have higher local pass rates than those in the middle of dense urban conurbations – particularly if there is a lower proportion of non-UK nationals taking tests.
Kelvin White seems to object to the ASA’s requirement for pass rates to be updated monthly with evidence, instead of just six-monthly.
An ASA spokeswoman said: “We received a complaint about advertising on the Kelvin White Driving School website which stated ‘you will also benefit from our first time pass rate of over 80% across the whole of the driving school’.”
Again, you can see the problem. You can’t just select your favourite pass rate and then use it as a marketing slogan for ever and a day. It will change – probably downwards in the real world, especially if you chose a high one at the start – and once you start boasting about the good results you commit yourself to having to publicise the bad ones if you don’t want to fall foul of the ASA. And that is quite right.