The Statistics Scam

I'm sure you remember my blog post Those Pesky Horizon Defenders, Well, the December 2019 judgment of Mr Justice Fraser in the Horizon IT case (link here) can help us expose another part of the Battlecrease scam, known as the Statistics scam.

For the benefit of the diary defender loon who posted in the Comments section of Numbers are hard": Really strange comment from you there David regarding maths and statistics. They are not even remotely the same field of numeracy. What were you trying to suggest? Did you get momentarily confused or did you geneuinely think they were one and the same thing? If you did, you have rather shown your ignorance and hugely undermined any argument you thought you were making in your earlier piece", I refer to the statement of Mr Justice Fraser in his judgment (para 819) that:

"Statistics is the branch of mathematics that concerns the collection, presentation and analysis of data."

We may at this point remind ourselves that Tom Mitchell admits to being hopeless at mathematics ("I've sadly never been any good at maths" he once posted) yet still thinks he understands statistics.

During the Horizon trial, as Mr Justice Fraser explained in his judgment, the word "robust" was "at the heart of the Post Office's defence of the Horizon system". Note the word "defence". We know all about Horizon defenders, of course.

In order to demonstrate that the Horizon IT system was robust, the Post Office's legal team called upon IT expert Dr Robert Worden, a leading Horizon defender, who devised an ingenious statistical argument to win the case for the Post Office. In fact, Dr Worden was the Horizon equivalent of Thomas Mitchell. Worden's statistical argument in a nutshell was that there were so many millions of successful transactions conducted by the Horizon IT system at post offices around the country each day, such that:

"If I selected a Claimant's branch and a month at random, then the chances of Bug A occurring at that branch in that month are only 16 in 8 million, or 2 in a million - an extremely small probability."

He referred to "the very small probability of any error in one months' accounts from a bug in Horizon" and said that the implication of this was that, "the accounts for any branch on any month are overwhelmingly likely to be correct".

In one simple statistical calculation, therefore, Dr Worden had successfully countered all the evidence of bugs in the Horizon computer system produced by the claimants, demonstrated the Horizon computer system to be robust and won the case for the Post Office, or so he and the Post Office hoped.

Unfortunately for him and the Post Office it was all absolute nonsense. Although Dr Worden is an experienced IT expert, with a PhD in Theoretical Particle Physics, anyone with half a brain could see what a lot of nonsense he was spouting. Many statisticians have since said so, and the judge completely demolished his statistical argument in the strongest possible terms.

Hence, Mr Justice Fraser said in his judgment:

"I consider this a wholly flawed methodology, and obviously so, and I reject it."

"Dr Worden’s reasoning is entirely circular."

"I consider that Dr Worden’s statistical exercise is nothing more than a superficial short cut which avoids a detailed technical analysis of the history of the Horizon system..."

"I reject his exercise... it is wholly flawed. I also reject the circularity of his reasoning, which lacks logic."

He referred to Dr Worden, "layering assumption upon assumption (and in places routinely misdescribing these assumptions as estimates", and described one of his key assumptions as "plainly invalid." He concluded that Worden's analysis was "so riddled with plainly insupportable assumptions as to make it of no evidential value", adding that, "It is the mathematical or arithmetic equivalent of stating that, given there are 3 million sets of branch accounts, and given there are so many sets of branch accounts of which no complaint is made, the Horizon system is mostly right, most of the time. It is a little more sophisticated than that, but not by very much."

The judge patiently explained at length where Dr Worden had gone wrong. It, frankly, makes an embarrassing read for the expert. The idea that the 600 or so sub-postmasters who were the claimants in the proceedings couldn't have suffered from Horizon bugs because other sub-postmasters hadn't reported problems, which was essentially what Dr Worden was saying, was so obviously flawed that it's hard to believe he seriously felt he could present it to the court.

It all reminds me of Tom Mitchell's desperate statistical argument to support the authenticity of the diary in circumstances where all the evidence tells us that the diary is inauthentic.

Tom Mitchell's statistical argument is just as flawed as Dr Worden's. Here's how he put it on 10th June 2020:

"Eddie Lyons has admitted that he was there at Battlecrease House (briefly) on Monday, March 9, 1992, the same day that Mike Barrett rang Doreen Montgomery 'out of the blue' asking if she'd be interested in Jack the Ripper's diary. There was no record of floorboards in Battlecrease House being lifted prior to that date (including the work that was conducted on the immersion heater in 1989), and nor (to our knowledge) was there ever previously a call to anyone suggesting they had the diary of Jack the Ripper (let's be more precise here, there had never - to our knowledge - ever been a call before which led to a published diary which survived at least 28 years in the very public eye). The chances of the floorboards coming up on the very day Barrett rang Montgomery is around 1-in-26,000 (the number of times it could have happened divided by the number of times it did happen). Speak to even an amateur statistician. Every single one of them will say that odds like those do not happen by chance alone. Something caused them both to happen that day. And I think you are seeing now how that 'something' has a name - Eddie Lyons."

The figure of 1 in 26,000 was later amended to 1 in 37,557 on the basis that there were 37,557 days between 12 March 1889 and 9 March 1992.

On 21 July 2021, he phrased the argument as:

"Two Maybrick-related events - one fairly prosaic but evidently very very rare (lifting of floorboards), the other very obscure and almost certainly bound to be unique (the call to the literary agent) happened on the same day after 37,557 days, and we have been debating the probability that they are unrelated (that is, resulting purely and simply from fickle chance that day)."

And then the following day we were told:

"Suffice it to say that the probability of those two events happening by chance after 37,557 days when either or both could have happened on any previous day is very very very very very obviously 1/37,557."

There are a number of fundamental flaws in Tom's argument.

The first is that it is based on factually inaccurate information. It's not true that, "There was no record of floorboards in Battlecrease House being lifted prior to [9 March 1992]" or, rather, to the extent that there is no record of floorboards being lifted prior to 9 March 1992, there is no record of floorboards being lifted on 9 March 1992 either. That's because there is no record in existence of floorboard lifting in Battlecrease at all. A register of the lifting of the Battlecrease floorboards does not exist; no record has ever been kept of this occurrence. It is inferred from the fact that floorboard protectors are on the timesheet record for this job that floorboards were lifted on 9 March 1992 but there's no "record" to this effect. We've been told by Paul Dodd that the floorboards were lifted in 1977 at a time when the building was gutted. So, if it can be said that there is a record of floorboards being lifted in 1992, there is also a record of them being lifted in 1977. This means that it's quite false for Mitchell to claim that there's no record of the floorboards being lifted before 1992. It's also entirely possible that the floorboards were lifted in 1946 when the building was rewired and in the 1920s when it was converted to electricity.

The second fundamental flaw is that in calculating the chances of Mike Barrett telephoning Doreen Montgomery on 9 March 1992, Mitchell includes in his calculation the years between 1889 and 1952 when Mike Barrett wasn't even born! How could Mike have telephoned Doreen, for example, on 9 March 1934? He couldn't, yet Mitchell's argument is, in effect, that Mike could have telephoned her on that day and the fact he did not is factored into his statistical calculation of probability.

Nor could Mike have been likely to telephone Doreen while he was young boy. We can surely rule out any year before 1970, when Mike was 18, as involving any possibility of him having contacted a literary agent to interest her in the diary of Jack the Ripper. We know that when he was 25 (in 1977) the floorboards in Battlecrease were lifted but he didn't call Doreen Montgomery then.

But there's certainly no physical way that Mike could have called Doreen or any other literary agent before he was born.

Mitchel has, in other words, taken what to him seems like a logical period for calculating probability and, in the process, created an utterly absurd period which makes no sense at all. How can any time before Mike was born have possibly impacted upon his decision to call Doreen on 9th March 1992? To switch it round, surely it wouldn't change the coincidence one iota if Maybrick had died in 1952 or in 1991, would it?

The third fundamental flaw is that Mitchell is forced to describe the mundane process of a lifting of the floorboards in Paul Dodd's residence by electricians on 9 March 1992 as a "Maybrick-related event". It's the only way he can express his statistical argument, otherwise he's talking about two unrelated events occurring on the same day, something which, of course, happens every single minute of every single day in every single week of every single year, But, if the lifting of the floorboards (assuming they were lifted) was a "Maybrick-related event", literally anything that happened in Battlecrease at any time between 1889 and 1992 must also have been a "Maybrick-related event". If Paul Dodd scratched his arse, it was a "Maybrick-related event". If he took a shit it was a "Maybrick -elated event". I mean, what's the difference between a Paul Dodd bowel motion within Battlecrease and a floorboard being lifted in the same building? Either they're both "Maybrick-related events", because they both occurred in Maybrick's old house, or neither of them is. The fact that, in Mitchell's argument, they must both be "Maybrick-related events" shows how ridiculous his statistical argument is.

The fact of the matter is that, on Tom's definition, there must have been "Maybrick-related events" occurring in Battlecrease every day of the week between 12 May 1889 and 9 March 1992. Not just arse scratching or going to the toilet but tidying, cleaning, redecorating, opening cupboards, entering rooms, switching lights on and off, cooking dinner, having a bath, making a telephone call, listening to the radio, watching the television, reading a book or newspaper etc. etc. More specifically, there were literally Maybrick tours going on in Battlecrease for many years. Yes, actual real "Maybrick-related events". So the chances of there being no "Maybrick-related events" occurring on the day that Mike telephone Doreen were, frankly, zero.

From a statistical perspective there's nothing special about the lifting of a floorboard which connects to the Maybrick diary. The Maybrick diary doesn't mention floorboards. There is no suggestion that the diary will be hidden under the floorboards. On the contrary, it is indicated that it will be placed somewhere it will be found. There's no evidence or reason to think that James Maybrick ever once lifted the floorboards or hid anything beneath them.

So by categorizing the lifting of the floorboards as a "Maybrick-related event", comparable to Mike touting the diary of Jack the Ripper written by James Maybrick (albeit that he didn't mention Maybrick's name on that day), Mitchell is adding onto what should be an objective mathematical formula a very subjective perception that there is some significance in the lifting of the floorboards which makes that event more important than anything else that ever transpired in Battlecrease between 12 May 1889 and 9 March 1992, and, indeed, is more important than a Paul Dodd arse scratch.

The fact of the matter is that the lifting of floorboards in an old house is probably happening every day of the week in every major town or city in the UK (and was happening every day in 1992), while there are (and were) also thousands of telephone calls happening every minute. So, by itself, the floorboards being lifted in an old house in Liverpool on the same day as someone in Liverpool was making a telephone call is not in any way statistically improbable. In fact, it's almost a certainty that those two things would have happened in any event, regardless of Battlecrease and Mike Barrett, on 9th March 1992.

Although he has claimed, laughably, that his analysis is "a strictly numerical analysis", Tom Mitchell has made an assumption that there is more significance in the lifting of the floorboards in Paul Dodd's house on 9th March 1992 than, say, the resignation from Parliament of Alan Amos MP after having been arrested for indecency with a man on Hampstead Heath, which also occurred on 9th March 1992. He has to, otherwise he would be compelled to say that Mike's telephone call must statistically be connected with Amos' resignation, with those two things happening on the same day. It demonstrates both the absurdity and falsity of Mitchell's claim that his analysis is "strictly numerical".

It's also a mistake to take two events that have actually happened (on the same day) and to argue backwards that their statistical improbability means that they are too unlikely to have happened by coincidence. You could equally do the same thing with the winner of this week's lottery. That winner's chances of winning would have been something in the region of 1 in 45 million, a number so large that the only rational argument must be that it didn't happen. Yet it did happen!

In language similar to Mr Justice Fraser, Jeff Hamm, who lectures on statistics, told Tom Mitchell on 22 July 2021: "...the improbability you're focused on is irrelevant and presenting it otherwise is a misuse of statistical probabilities with regards to research and investigations."

CONCLUSION

Just like Dr Worden in the Horizon case, Mitchell thinks he's solved the evidence problem (i.e. in his case, the complete lack of evidence demonstrating that the diary was found under the floorboards of Battlecrease) by producing a nonsensical statistical analysis. In this, like Dr Worden in the Horizon case, he fails entirely.

This, ladies and gentlemen, is the Statistics Scam exposed.

LORD ORSAM 13 August 2024