It was an Imperial College computer model that forecasted 500K deaths in the UK (and 2.5 million in the US) should policymakers pursue a “herd immunity” approach (a la Sweden), that influenced them to reverse course and go full lockdown instead. The model was produced by a team headed by Neil Ferguson, (who recently resigned his post advising the UK government when it surfaced that he was himself violating lockdown directives by breaking self-isolation for dalliances with a married woman).
The source code behind the model was to be made available to the public, and after numerous delays and excuses in doing so, has finally been posted to GitHub
A code review has been undertaken by an anonymous ex-Google software engineer here, who tells us the GitHub repository code has been heavily massaged by Microsoft engineers, and others, in an effort to whip the code into shape to safely expose it to the pubic. Alas, they seem to have failed and numerous flaws and bugs from the original software persist in the released version. Requests for the unedited version of the original code behind the model have gone unanswered.
The most worrisome outcome of the review is that the code produces “non-deterministic outputs”
Non-deterministic outputs. Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.
This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all – as the field of psychology has been finding out to its cost. Even if their original code was released, it’s apparent that the same numbers as in Report 9 might not come out of it.
The documentation proffers the rationalization that iterations of the model should be run and then differing results averaged together to produce a resultant model. However, any decent piece of software, especially one that is creating a model, should produce the consistent results if it is fed the same initial data, or “seed”. This code doesn’t.
“The documentation says:
The model is stochastic. Multiple runs with different seeds should be undertaken to see average behaviour.
“Stochastic” is just a scientific-sounding word for “random”. That’s not a problem if the randomness is intentional pseudo-randomness, i.e. the randomness is derived from a starting “seed” which is iterated to produce the random numbers. Such randomness is often used in Monte Carlo techniques. It’s safe because the seed can be recorded and the same (pseudo-)random numbers produced from it in future. Any kid who’s played Minecraft is familiar with pseudo-randomness because Minecraft gives you the seeds it uses to generate the random worlds, so by sharing seeds you can share worlds.
Clearly, the documentation wants us to think that, given a starting seed, the model will always produce the same results.
Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.
In one instance, a team at the Edinburgh University attempted to modify the code so that they could store the data in tables that would make it more efficient to load and run. Performance issues aside, simply moving or optimizing where the input data comes from should have no effect on the output of processing, given the same input data. What the Edinburgh team found however, was this optimization produced a variation in the output, “the resulting predictions varied by around 80,000 deaths after 80 days” which is nearly 3X the total number of UK deaths to date.
Edinburgh reported the bug to Imperial, who dismissed it as “a small non-determinism” and told them the problem goes away if you run the code on a single CPU (which the reviewer notes “is as far away from supercomputing as one can get”).
Alas, the Edinburgh team found that software still produced different results if it was run on a single CPU. It shouldn’t, provided it is coded properly. Whether the software is run on a single CPU or multi-threaded, the only difference should be the speed at which the output is produced. Given the same input conditions, the outputs should be the same. It isn’t, and Imperial knew this.
Nonetheless, that’s how Imperial use the code: they know it breaks when they try to run it faster. It’s clear from reading the code that in 2014 Imperial tried to make the code use multiple CPUs to speed it up, but never made it work reliably. This sort of programming is known to be difficult and usually requires senior, experienced engineers to get good results. Results that randomly change from run to run are a common consequence of thread-safety bugs. More colloquially, these are known as “Heisenbugs“.
Another team even found that the output varied depending on what type of computer it was run on.
In issue #30, someone reports that the model produces different outputs depending on what kind of computer it’s run on (regardless of the number of CPUs). Again, the explanation is that although this new problem “will just add to the issues” … “This isn’t a problem running the model in full as it is stochastic anyway”.
The response illustrates the burning question: Why didn’t the Imperial College team realize their software was so flawed?
Because their code is so deeply riddled with similar bugs and they struggled so much to fix them that they got into the habit of simply averaging the results of multiple runs to cover it up… and eventually this behaviour became normalised within the team.
Most of us are familiar with the computing adage, “Garbage In/Garbage Out” and the untrained reader may think that’s what being asserted in this code review. It isn’t. What’s being asserted is that output is garbage, regardless of the input.
In this case, the output we’re experiencing as a result is a worldwide lockdown and shutdown of the global economy, and we don’t really know if this was necessary or not because we have no actual data (aside from Sweden) and severely flawed models.
Read the entire code review here.
To receive the #AxisOfEasy weekly tech digest, covering all issues privacy, security, censorship and surveillance, join here, or follow us on Mastodon or Twitter
Holy clown shoes. I’ve seen some pretty bizarre code come out of academia, but this is a train wreck.
I am particularly amused the sleight of hand use of “stochastic”. Reminds me of a time I referred to “complex heuristics” used in a decision process when it was actually just my opinion. The difference is that I was clearly joking but these idiots actually believed their own BS.
It seems that lockdown, while drastic, was probably good policy anyway but the fog created by this undermines good policy and good faith.
The models of this Lyssenko have already killed 6 million cows and destroyed the beef industry in the UK.
Almost everyone in the US thinks lockdown is absolutely necessary and are terrified of it being lifted when in fact it shouldn’t been done in the first place. Medical Date shows 80% have it and get over it w/o needing to be hospitalized, especially if they are treated with Zinc and Hydroxychloroquine which only takes 7-10 days w/o any drug toxicity. About 20% need hospitalization and 75%-80% of those patients resolve w/o problems. The overall rate of Deaths is 1-2% or less and those patients have various risk factors like Diabetes, Obesity, Cardiovascular Disease, advanved age or suppressed Immune Systems which is exactly happens with yearly Flu and we do not shut down for the Flu. There are at least 4 drugs that work so the Swedish method is best and accept the fact the vulnerable have a higher possibility of Dying just like with the Yearly Flu. This is the way to achieve Herd Immunity which is the Goal. GIGO IS THE RULE.
idiot
Not ‘almost everyone’. A lot of us think our governors are full of sh*t.
What exactly does the “MD” at the end of your name stand for? Because you obviously aren’t a medical professional of any type or training. Every single sentence in your little tirade contain fundamental errors in terminology, basic facts, and human physiology. Even if you rewrote your post and corrected the grammar and spelling so that your ignorance isn’t highlighted by your inability to avoid run-on sentences, there would still be an error in every sentence. Didn’t your 5th, or even 3rd, grade teachers instruct you on how to properly form a sentence? Have you heard of the great technological feat they call “spellcheck?” And just because the Twit in Chief insists on conflating truth with erroneous capitalization of words, doesn’t mean that it is any less ridiculous and self-sabotaging than when any other garden-variety idiot does it. Come on, “Deaths,” “Dying,” “Immune Systems,” “Medical Date?” Seriously? I agree that we should never have gone on lockdown, but my reasons for believing that stem from actual medical training, well-established facts, and recognition that the lockdown most likely did reduce the numbers of deaths from SARS-CoV-2/COVID-19 so far. A death rate of 1-2% (an estimate that could be wildly inaccurate) for an infectious disease places that disease in the 99% percentile for lethality in humans. Seasonal influenza (not “Yearly Flu”) has a mortality rate 100 times less than 1-2%. Never mind that SARS-Cov-2 is orders of magnitude more contagious and infectious (these terms are not synonymous; look it up) than seasonal influenza. The only known infectious diseases more contagious than SARS-Cov-2 are smallpox, measles, and pneumonic plague. Plague is caused by a bacterium that is highly susceptible to almost all antibiotics, smallpox was eradicated almost 40 years ago by vaccination, and measles was almost eradicated by a vaccine that is extremely effective, and lifelong. Which means COVID-19 is currently the most dangerous infectious disease infecting humans. The 1-2% death rate that you dismiss so casually translates to 70-150 million deaths worldwide in the first year alone. There are no current treatments or vaccine candidates for it. Hydroxychloroquine is like every other pharmaceutical: it always has side effects and adverse reactions (and is one of the most toxic drugs outside of cytotoxic chemotherapy agents). Zinc is pointless, and can’t reduce the side effects of hydroxychloroquine. Nevermind that hydroxychloroquine has never been demonstrated in *any human trial whatsoever* to do anything other than increase mortality when taken for COVID-19 treatment or prophylaxis. And because of idiots like you, I have patients who desperately need hydroxychloroquine to manage lupus, rheumatoid arthritis, and other autoimmune conditions, who can’t find a pharmacy with it in stock. BTW: just because another human being has a medical condition that increases their risk of dying from an infection, doesn’t mean their lives are any less valuable or worthy of dignity, respect, and medical care than someone in better health. I have a genetic immunodeficiency and other comorbidities that place me at the greatest risk of dying from COVID-19. I also grew up in the 4th poorest country in the world, so I saw, every day, how lethal and devastating the economic disaster we’ve caused is already killing more people than COVID-19 could have. And herd immunity is great, when you’re dealing with an infection that causes immunity in survivors, which increasingly looks to not happen with COVID-19. So, like you concluded: “GIGO IS THE RULE!” idiot.
There is no way to determine the actual death rate of those infected since vast numbers of people never have severe symptoms and never get tested. Both my 56-year-old brother with Celiac and my 90-year-old father-in-law both had Covid-19 but never had life-threatening symptoms, indicating that perhaps it is not as deadly as you seem to believe. So far there is no conclusive evidence that this is worse that the ’58 or ’68 influenza pandemics, neither of which caused a global shutdown.
This is not so much a case of GIGO as AIGO (Anything In, Garbage Out).
I cannot comment on the efficacy against Covid-19 of Zinc+Hydroxychloroquine. However there is a small group of men (mostly of middle-eastern or African origin) who carry a G6PD enzyme variant that can cause a life-threatening reaction to any quinine derivative. Make sure you get your G6PD status checked before accepting the treatment.
Medical data from people who have collected data shows that possibly 3% have it.
To say that “Medical Date shows 80% have it…” are the words of a non-MD non-English speaker.
Yeah. The very fact that this code review is published on a site called “lockdown sceptics” pseudonymously pretty much sums it up.
I know you know it all ( I mean, I read the weekly newsletter) but man, you’re pushing some dangerous ideas lately.
I’m sure the reviewer had better things to do than have their life ruined by the mainstream press for being heretical. Since the remarks are all about an open source repository that is there for all to see, if anything outlined in the review is wrong it would be easily refuted.
The fact that the review was run on a site called lockdown skeptics is immaterial, it’s actually a logical fallacy (“genetic” or “poisoning the well”). We sure weren’t going to see this covered by CNN or Vox, or even VICE so where else is it going to publish? Big Tech is literally and actively suppressing any dialog that doesn’t conform to official sources.
We’re now living in a world where our entire economy and social structure has been essentially shut down and is being ruled by decree for about 8 weeks now, and nobody is permitted to question the rationale or the models used to arrive at this mode of governance.
So you tell me, what’s dangerous?
Hey. Thanks for your work!
How can I please get in touch with someone to submit a model that is being used here in NZ to get 100% of the population vaccinated? I would love to have some insights from the coding community on this one https://www.tepunahamatatini.ac.nz/2021/06/30/a-covid-19-vaccination-model-for-aotearoa-new-zealand/
We don’t know the denominator except in contained outbreaks (meat plants etc.). In these instances the hospitalization rate is under 2% and the death rate less than 0.2 %. Still a lot of people when spread over millions.
Those on ventilators have greater than 50% mortality. The economic cost per life saved during this pandemic is astronomical and would never be considered under normal circumstances in a publicly funded health care system.
“Still a lot of people when spread over millions”, yes. But not remarkable when considered in the context of regular flu.
According to a recent New England Journal of Medicine editorial (lead-authored by Anthony Fauci), when you consider statistical apples-to-apples, “…the overall clinical consequences of Covid-19 may ultimately be more akin to those of a severe seasonal influenza (which has a case fatality rate of approximately 0.1%) or a pandemic influenza (similar to those in 1957 and 1968) rather than a disease similar to SARS or MERS, which have had case fatality rates of 9 to 10% and 36%, respectively.”
For this we’re suppressing dissent and tyrannizing a world?
I’m curious as to whether we’ll see additional reviews of the code, both by individual programmers and by institutions (universities and government entities). You’d think that lots of people, both with an ax you grind and without) would be chomping at the bit to have a look at it.
Given the questionable quality of the Imperial College code, I’d also think that there would be various groups (like, I dunno, the CDC and the WHO) working around the clock to either improve it completely replace this software.
Thanks, Mark. I look forward to hearing more in the future.
— Considering the damage that’s been done, the lies that have been told, and the length of time that this has gone on, this is a true scandal; this ought to be — but won’t be — a career-ending scandal for more than a few people in positions of public trust.
— I notice that the lynch-mob, wolf-pack, fellow-traveler media is ignoring this with great vigor
— Microsoft is complicit in the coverup? Somehow I’m not at all surprised.
— This reminds me of Michael Mann’s now-discredited “hockey stick” global-warming model scandal.
Final thought: Is a scandal only a scandal if the media says it’s a scandal?
And the current virus is only virus because the mass media said so. If the satanic mass media did not conspire to overhype the plandemic (on behalf of the sinister elites bent on ushering the NWO ) – the world would not know that it had a problem. All personal symptoms would be attributed to a usual seasonal “nasty bug going around”.
So if it really is a problem do this.
Write your local representative to the legislature.
Ask them to have a review of the C++ code performed.
Ask them to have any policies initiated by this data reviewed.
Ask them to override the governor’s overreach that is resulting in a lockdown based upon faulty data.
Have your friends do the same thing.
I am not sure why they would have been multi-thread optimising by hand? Well tested compilers usually do a better job than all but the most skilled hand-coders (which most academics are not). The compilers are designed to avoid the pitfalls.
I remember debugging a monte-carlo scientific code that was producing “exploding” crystals on an IBM 370 computer, but not on a different make. It turned out that the original programmer had done some arithmetic in the wrong order, and failed to appreciate that floating point rounding errors accumulate significantly over tens of thousands of steps. The two computers used different hardware rounding algorithms. The non-IBM computer’s method suppressed the error accumulation by a couple of orders of magnitude. Changing the order of calculation fixed the code so that the results were inidstinguishable on the two computers (i.e. teh IBM crystal no longer “exploded”.
I would be willing to bet that the coders of the Imperial College model did not have a good grounding in the subtleties of numerical computing errors. When you do algebra, you can assume infinite precision, so two ways of getting an answer will be identical on paper. However, when you are coding, different routes can produce vastly different errors. For example, you should never compute two big numbers, if the answer you want is their difference. You could easily find that the output is just rounding noise.