This is an old story. DSA is now DVSA.
A story appeared a few days ago on the BBC website about a driving examiner in Devon, who claimed she was bullied over her pass rate.
Nicola Bentley-Lovell made a claim of constructive dismissal against the DSA. She’d worked as an examiner for seven years up until October 2010 but resigned after she allegedly had to take a lot of “bullying and harassment” over her pass rate.
She said that over a six-year period she underwent 165 internal check tests (the normal check rate in that period would have been a maximum of around 50 per examiner).
She claims that her managers “urged her to increase her pass rate” as it was below “acceptable variance”.
You can read the full story in that link – it is bulked out with a variety of claims that are not directly connected with the issue of variance.
At this point, the forums lit up with the usual crowd of fossils and know-it-alls – idiotic claims about people always failing tests on a Friday, accusations that they get cash bonuses for meeting targets, and so on. Hard to believe many of these people can get away with calling themselves “professionals”.
What shut them up rather quickly was the subsequent story a few days later – also on the BBC – that the examiner had lost her case of unfair or constructive dismissal.
Now, I’ve written about variances before. There was a case where a Scottish ex-examiner was moaning about how his bosses wanted his pass rate to come down! He also had a gripe over “variances”.
What is clear is that a lot of people – examiners like these two, and plenty of ADIs who persist in their archaic careers solely to hate the DSA – do not have a clue what variances actually are.
If you toss a coin, it is 50:50 whether it will be heads or tails. In theory, if you toss it a hundred times, you should get 50 heads and 50 tails. In reality, you might get 45 heads and 55 tails – or the other way round… or anywhere in between. Or even a bit different.
What you won’t get is 80 heads and 20 tails. And even if you did, you wouldn’t get the same again, and you certainly wouldn’t get it if you upped the tosses to 1,000 or 10,000.
The same is true of examiners. No matter what happens, in theory only a certain percentage of test candidates will pass, and it is not 100% of them. So if you have on perfect examiner conducting all tests, you would end up with a pass rate somewhere around 40%. That’s just the way it is – that 40% is the equivalent of the 50:50 of coin tossing.
Arguably, if all ADIs were perfect and taught to a perfect standard, and this one perfect examiner took all the tests, then the pass rate would theoretically be 100%. But you don’t have perfect ADIs, you don’t have perfect pupils, and you don’t have perfect examiners.
What you do have are typical human beings who are affected by typical human defects. Variances, if you like. And that’s where the 40% comes from.
Now, if you tossed that coin a hundred times and it came down 80 heads, would you think there was something wrong with it? Well, I would. And if the same thing happened for the next hundred tosses, I’d be certain. And if it kept happening, I’d use a different coin next time.
But the problem appears to be that some people wouldn’t see this as an issue and would reject any attempt to fix the coin so that it came down 50:50 like it should do. And this is exactly what keeps happening with these examiners who keep going on about variances, and the ignorant ADIs anxious to believe the worst.
If an examiner has a pass rate which is not the equivalent of 50:50 for a coin toss test (or 40% in their case), or within reasonable limits (95-105% of that 40%), then they are not performing to the required standard – just like the coin that keeps giving 80 heads in every hundred tosses!
Both examiners in these publicised cases appeared too stupid to realise this simple fact.
If someone is on your case for six years – giving you every opportunity to examine what you do – you have to be a very special kind of person not to be able to realise something is up. And you need to be extra-special to conclude that you – out of hundreds and hundreds of other examiners – are not doing something wrong.