The first confirmed COVID-19 case in the Netherlands occurred on February 27, 2020. Over the following days and weeks, COVID-19 numbers increased rapidly as did the number of comparisons with Italy, some are mentioned here in Dutch. Italy had seen its first confirmed COVID-19 case earlier, on January 31, and by the early days of March had accumulated thousands of confirmed cases and hundreds of deaths, while daily numbers were rising with no end in sight: the famous exponential increase – or perhaps more precisely the initial phase of logistic growth, which closely mimics exponential growth.

The RIVM, legally independent of the health ministry under which it resides, took to the task of assessing and managing the Dutch situation. It judged that a major outbreak in the Netherlands was unlikely, but did make some minor suggestions as numbers kept rising, like the March 9 suggestion to suspend shaking hands. It also communicated that the Italian experience was uninformative for the Netherlands. The number of cases – and deaths – rose rapidly.

Denmark, which like the Netherlands had seen its first confirmed case on February 27, held the same view in the early days of March: a major outbreak was unlikely in Denmark. But only days later it recognized its mistake, changed its judgment to a major outbreak being a virtual certainty. Denmark went into full lockdown on March 11.

The Netherlands continued on its path, the Prime Minister giving a rare address to the nation on March 16, stating the purpose of letting the virus circulate “in a controlled manner”. The RIVM persisted in its judgment that a comparison with the Italian situation was uninformative, peddling its own statistical and mathematical modeling instead, which predicted no real problems.

I got worried as it became increasingly clear to me how the RIVM was approaching this problem – and that it persisted in doing so. On March 21st I wrote to family and friends that the RIVM was getting this fundamentally wrong, that I saw no way in which COVID-19 would not overwhelm healthcare capacity in the Netherlands within weeks, and that they should be careful. What else could I do? The next day, March 22nd, a newspaper article appeared that showed, yet again, why the RIVM was wrong: Among others, its modeler-in-chief explained “why back-of-the-envelope calculations do not work”, as the headline proclaimed, meaning comparisons with Italy or other simple extrapolations of exponentially growing numbers. Only the models were to be trusted.

Now it is true that models provide insights much beyond common sense or human intuition. But all models have one thing in common: if the analyst is guessing what to model, then the output is just guesswork too. Models return their input, but in a different form, and how the form changes depends on the model. This means that models can be wrong for two reasons. First, they might fail in the way they convert input to output, meaning that they could convert correct input to wrong output. Statisticians sometimes call this model uncertainty. Models might also fail because the input is wrong: converting input to output in the right manner is of no help if the input cannot be relied upon: input uncertainty. More directly phrased: garbage in, garbage out.

Now let us assume, for the sake of the argument, that the RIVM modeling was exquisite and if the RIVM had known how the virus was transmitted, which symptoms patients had, and so forth, their predictions would have been spot on. Knowledge about the virus, however, was limited. Large scale testing did not yet exist, and the novel corona virus was, well, novel. Limitations in testing capacity led to changing recommendations on who to test and to artificial deceleration in case number increases. As a result, models were models of test regimen perhaps more than of actual cases. The RIVM’s insistence that all COVID-19 patients had a fever – and that therefore only those with a fever should be tested – further shows just how limited the understanding of the corona virus was at the time. Hence, clinging to models and dismissing real-world data as misleading was, with the information available at the time, a mistake. The opposite would have been closer to the mark.

Ironically, one day after the publication of the aforementioned surprising article the Netherlands finally surrendered and went into full lockdown. It saw its intensive care capacity exceeded throughout the end of May. Many an elderly home saw its population halved. The Italian scenario had arrived.

It is not the point to argue for or against lockdowns. These are difficult decisions with important ideological components. There is also no denying that resources like facemasks were scarce, and that societies tended to be overall poorly prepared for major pandemics. Indeed, the Italian region of Lombardy, the focal point of the pandemic at the time, was relatively well prepared, while the Netherlands were relatively poorly prepared, with per-capita beds in intensive care units (ICU) extremely low for European standards, for example at less than 20% of the German number. This was also true for the per capita number of acute care beds in the lower regions and should have been another reason for caution.

It is also not the point to argue for simple extrapolations or comparisons as a general source of information when deeper understanding is available. Societies need science, and statistical modeling often provides a precision that back-of-the-envelope calculations cannot give. The problem is rather that such deeper understanding was not available. In such cases, a ponderous discourse on statistical modeling obscures the lack of knowledge underlying these models. The RIVM were largely modeling their own assumptions, which had not been tested, and which often resulted from circular reasoning. For example, the RIVM believed that corona always causes a fever, consequently recommended to test only patients with a fever, and then looked at the data to conclude that all corona patients have a fever. While older patients especially, less prone to produce a fever, remained undiagnosed.

So why would a number of trained professionals overestimate and/or overstate the utility of their modeling? What can we learn from this case? Had the early COVID-19 situation been described to the RIVM as a theoretical case, for example in the form of an exam question, the RIVM would presumably have gotten it right and have criticized such focus on modeling in situations fraught with uncertainty. More precisely, they would have emphasized that such models have enormous utility in exploring the vast space of possible scenarios, but that with limited information available, only a very low probability could be assigned to each of those scenarios individually, meaning that no firm prediction could be made from these models with the information available at the time. Such uncertainty would then have sent the RIVM looking for real-world evidence, i.e. the experience in Italy, to give further guidance as to which scenario might play out. The Italian scenario, the RIVM would have concluded like its Danish counterpart, was a very distinct possibility.

So why did the RIVM not stick to science, especially since before the novel corona virus came to the Netherlands, scenarios of a major outbreak do seem to have circulated within the RIVM? Various explanations are possible. First, there is the saying that if you are a hammer, the world looks like a nail. If you are a modeler, your first instinct may very well be to draw up a model. Very often – but not always – that instinct is right.

The RIVM may also have felt an expectation as scientists, to produce a “scientific” result. From a scientist, one does not expect the answer “I haven’t the foggiest”, so a pressure may have been felt to produce something, anything ̶in much the same way an old-fashioned physician may not admit to being out of their depth – and certainly not to a patient!

Third, the structures of the RIVM had been shaped to address minor outbreaks of near-eradicated pathogens such as the polio virus, or to trace relatively pedestrian epidemics like the yearly flu. This did not prepare for a major outbreak of a relatively unknown virus like the novel corona virus. The ways of thinking and decision making were unprepared. And escalation to a higher level of abstraction seems to have been insufficient, if not absent. Finally, pressure came from politics. While the RIVM is nominally politically independent, in practice this is more difficult, and it is not without reason that new employees receive training in dealing with government officials although whether that training is put into practice is, of course, another matter entirely. Politics can be quite dominant in the Netherlands, and it is not always easy to retain professional independence. Tellingly, an awkward spelling mistake made by a government official in an email commenting on Outbreak Management Team (OMT) draft meeting minutes emerged in the final RIVM version of those meeting minutes. The OMT is an independent RIVM-mandated expert group. OMT members have stated they were not informed of such interference, but to have suspected it because they felt that the minutes often did not reflect the discussions.

Misunderstanding and/or misrepresenting the nature of mathematical modeling is not the only thing the RIVM got wrong. Its epidemiology section published a study selecting infected adults, observing them and their families. From this study, the team of epidemiologists concluded that adults are more often infected than children, and that COVID-19 is more often transmitted from adults to children than in the opposite direction. Thinking that I must have missed something, I wrote to the RIVM’s lead author to confirm what had been done. Surely there was a misunderstanding? But no, this was exactly what had been done, the lead author confirmed. To my follow-up question whether this was a fit-for-purpose study, justifying its conclusions, I received… no response. I therefore noted my reservations in a letter to the editor, which was published just a day before a team of renowned epidemiologists voiced similar concerns.

The RIVM further advised that flying during the pandemic was safe because it was economically unviable to fly with near-empty planes (a rather interesting causal statement), and that it was up to the aviation industry to devise rules that would guarantee passenger safety (a political statement). As mentioned, they further recommended testing only in case of fever… to conclude that all people infected with the corona virus have fever; while low availability of facemasks played a rolein its assessment that facemasks were not necessary to prevent transmission.

It is difficult to view these mistakes as isolated cases. Unfortunately, the most realistic interpretation is that the RIVM lacked essential expertise to do good studies quickly, regularly exceeded its mandate, showed an undue sensitivity to politics, and lacked the capability and/or independence to drag itself out of an unproductive loop.

A clear delineation of competencies – and pursuing those competencies vigorously – is one of the great pillars of the modern democratic state. The RIVM gives science-based medical and epidemiological advice – period. If schools should reopen to avoid learning delays, which is a very good argument, let that discussion be had. But do not patch together a study so glaringly biased. If a lockdown is seen to unduly limit personal freedom, then let’s have that discussion too. But do not insist that a major outbreak is unlikely on the basis of poorly informed models that contradict real-world evidence. And so forth.

Focusing on its legal role and beefing up on problem-analysis skills while steering clear from politics should get the RIVM in a better place. This is a change in capabilities and thinking. Politicians in turn should not hide behind science when making decisions. They should heed scientific advice but be honest about the limitations of science and the ideological components of decisions.

Separation of power and competencies is one of the great buttresses of effective, democratic government. It should be adhered to at all times – especially when it is most needed: under pressure.