whydoitweet

I'm a Lecturer in Computer Science at Brunel University London. I teach games and research simulations. Views are my own :).

Hi everyone. I haven't been reviewing any models as of late, because I've been asked to develop an agent-based model for Covid-19 from the ground up, given the emergency situation here in London (and in many other places). I'll keep you updated on this progress if time permits, but until then I guess there will be only sparse updates here.

Nevertheless, I found a very good critical article about using models in the Guardian. You can find it here: https://www.theguardian.com/science/2020/mar/25/coronavirus-exposes-the-problems-and-pitfalls-of-modelling

My only criticism of that article is that doesn't say much about how models can also help communities in ways other than just providing forecasts. For more information on that, please have a look here in the article “Why model?” instead :).

Credits

Header image courtesy of pixnio.

Today I'm reviewing what is one of the most influential Covid-19 simulation project today: the simulations by Neil Ferguson et al. from Imperial College London.

It is not often that simulations (or models) get referred to in Prime Ministerial Press briefings, but during this pandemic that clearly has changed.

Photograph by Alberto Pezzali from Reuters.

Every day we now see new measures being enacted in the UK, and Prime Minister Boris Johnson on stage flanked by scientific and clinical experts. And many of the recent changes made by the UK government are based on a microsimulation report from Imperial College London, which you can find here:

https://www.imperial.ac.uk/news/196234/covid19-imperial-researchers-model-likely-impact/

The PDF report (which is linked to in the above article) stretches to 20 pages, and has clearly been written with mostly a scientific/policy audience in mind.

The approach they use is called an “individual-based stochastic simulation model”, which is a slightly ambiguous phrase. It could mean that there are individual entities that behave in a stochastic way (i.e., a type of agent-based model like the one I made a tutorial for earlier) or it could be that there are stochastic equations that mimic the individual behavior. I am not certain at this stage, but I suspect it could be the latter. For sure the simulations are discrete in time, as this is clearly mentioned in an earlier paper.

In terms of claims the report mentions a wide range of things, but the key take away is this:

“(...) policy strategies which aim to mitigate the epidemic might halve deaths and reduce peak healthcare demand by two-thirds, but that this will not be enough to prevent health systems being overwhelmed. More intensive, and socially disruptive interventions will therefore be required to suppress transmission to low levels. It is likely such measures – most notably, large scale social distancing – will need to be in place for many months, perhaps until a vaccine becomes available.”

The whole paper hinges on two strategies, each of which comes with a set of measures. These are mitigation, where the aim is to stop the epidemic in its tracks entirely, and suppression, where the aim is to allow the epidemic to proceed in a controlled fashion.

The code and relevant data sources are not yet public, but in direct communication with the development team (yes, I e-mailed them and got a response(!)) I found out that the code will likely become available in the coming weeks, as they are documenting it at the moment.

But there is no sense in making a review with full information in three weeks if the government is using the outputs to make recommendations today. So please consider this as a Best Effort review, given the available information I have today.

(if new information or details emerge, I am happy to update this review, by the way)

In terms of individual measures, they make forecasts for the following five interventions, and combinations of those:

If you have been following the government speeches, then these measures may look vaguely familiar... ;).

As for the key results, I'll not fixate on the death estimates (these rely heavily on the mortality rate, which is not that clearly known yet), but instead look at the key findings of the different runs.

Here is an overview of the number of critical care beds needed, given different combinations of measures used in a mitigation context.

And here is the same graph, but for measures used in a suppression strategy:

Yes... those green and orange lines sure are a lot lower than anywhere else. The red line is the surge critical car bed capacity, by the way.

Anyway, I hope this gives you an overall impression of the work, so let's move on to

The Review

In the review, we cover four areas (as mentioned before) in order: Robustness, Completeness, Quality of Claim, and Presentation. I maintain this order, because the last one is pointless if the first one isn't there, and the third one is not a total disaster if the underlying model is at least of some use. So let's have a look at the

Robustness

This is a simulation from a group that has a clear and strong track record in epidemiological modelling. For instance, the 2008 paper I mentioned earlier, and from which this has been derived, has been cited a lot of times, and was published in a very high profile scientific journal. In the report itself they also meticulously justify the choice for each parameter, and document the way they construct the simulation reasonably systematically.

However, many aspects of Covid-19 are still uncertain, for example we still do not exactly know how many other people an infected person is likely to infect. This number is called the R0, so if R0=2 that means that every patient will on average infect two other persons.

Because of this uncertainty it's important that the researchers do a large number of runs with different settings. The group did that to some extent: to take R0 as an example, they chose to run simulation using an R0 number of 2.2, 2.4 and 2.6. This is a subset of the range presented across all researchers, but matches roughly with an assessment published in Nature yesterday. But other aspects are not covered as systematically. For instance, what happens if the compliance for the different measures is different than expected, or if the asymptomatic people are more or less infectious than they currently assume?

People especially know little about the infectiousness of asymptomatic people, because in most countries these people do not even end up being tested. So it is important to test how the model outcomes change when we change any assumption on that.

Lastly, in Figure 3 you can see that the orange and the green lines, which represent measures the government are taking now, have low values and show some errative jiggly patterns. We do not know what causes these distortions, but it the low values imply that the stochastic (random) components of the simulations probably have a bigger effect here, and that these lines are more sensitive to slight changes in the underlying assumptions. In my opinion it would be good if the researchers can analyse these sensitivites more systematically, and fit error bars on these lines, which are now literally of life-and-death importance.

Robustness rating: 3.75/5. The group justifies the assumptions in their simulations well, there is no obvious bias I can tell (but I'm not an epidemiologist) and the writeup clearly shows all the details. However, I take off half a point because I cannot scrutinize the source code or the data, and half a point because they easily could have tested the sensitivity of their weaker assumptions more systematically, and it's not clear to me that they did.

Lastly, I take off 0.25 points because I believe the researchers specifically need to better analyse the sensitivity of their model in the regime that we currently pursue (orange and green lines in Figure 3). The squiggliness of the lines indicate that random effects may be more dominant there than we'd like.

3.75 out 5 is a critical score, but please do note that I deliberately set the bar so high, because this model is used for government decision-making.

Completeness

This is a pretty complete work. The researchers examine the US and the UK, look beyond the first wave of the epidemic, and investigate a very comprehensive range of measures, categorized across two strategies (mitigation and suppression) that are indeed widely pursued now. We also have to appreciate that this work has been done in a very short time span, and given those constraints, I can say that this work does an excellent job in terms of completeness.

However, I cannot give a perfect score for two reasons:

  • They did not provide the source code and input/output data to the public, which we academics are normally expected to do so in non-commercialized cases.
  • They didn't present results for all combinations of measures.

The first point I already spoke about earlier, so here I'll discuss only the second point. To illustrate this point, let me share one of the key results tables, about mitigation measures:

Table 3 from the report. Higher numbers imply a greater reduction, and are therefore more positive. The rectangular highlight is added by me.

Here each column indicates a measure. These are respectively:

  • PC (close schools for 3 months)
  • CI (isolate infected people at home for 3 months)
  • HQ (quarantine households of infected people for 3 months)
  • SD (do social distancing for 3 months)
  • SDOL70 (do social distancing for people over 70 for 4 months)

The researchers present 7 combinations of measures, but of course many more combinations are possible, including non-sensical ones like HQ without CI, but also very sensible ones like only doing SDOL70 or PC+CI+HQ+SD.

Currently these other combinations are not in the main report, nor in any of the appendices, so it would be good if these can be added.

It would in particular be good if these can be added, because there is one other issue in this table. If you look at the rectangular region in the Figure above, then you can see the change of deaths predicted for two different sets of measures (CI+HQ+SDOL70 and PC+CI+HQ+SDOL70). So essentially, the right column in the rectangle has the same set of measures as the left column, but also includes school closure. Now according to the results, the amount of deaths we prevent drops from 49%-50% to 19-32% when we close the schools in this context.

I could not find an explanation for this strange artifact in the results, so I think this needs to be added (or the result corrected?). Also, because not all combinations of measures are presented here, I cannot clearly tell what causes this strange result. However, we do know that school closures alone result in an expected 2-4% decrease in deaths (see the leftmost results column labelled 'PC').

Completeness rating: 4/5. The researchers really went to great lengths to try and be complete, given the time pressure. However, I need to subtract 0.75 point because they did not present the results from all relevant combinations of measures, even though there is clearly at least one strange effect going on in the forecasts. Also, I subtract 0.25 point for the lack of research data and source code (small penalty, because I already included it in robustness too).

Quality of Claim

As a reminder, the main claim in the report is that:

“(...) policy strategies which aim to mitigate the epidemic might halve deaths and reduce peak healthcare demand by two-thirds, but that this will not be enough to prevent health systems being overwhelmed. More intensive, and socially disruptive interventions will therefore be required to suppress transmission to low levels. It is likely such measures – most notably, large scale social distancing – will need to be in place for many months, perhaps until a vaccine becomes available.”

Although the simulations are not perfect (almost no simulations are, by the way), the claim is so overwhelmingly supported by the simulation results that it is likely to hold despite the simulation issues we discussed earlier. The researchers also go to great lengths in the report to put their claims in the right scientific contexts, highlighting all the ifs and buts of the matter carefully.

What is a little less certain from the simulations is that the suppression strategy prevents an overload of the intensive care units. And, indeed if you read carefully you'll notice that the researchers do not claim that suppression strategies will entirely prevent overwhelming intensive care units. And, given the limitations of the simulations they used, that was a sensible thing to do.

So overall, the researchers could be overly negative about mitigation strategies, but the claims as a whole are definitely proportionate.

Quality of Claim: 4.75/5. Small parts of the claim refer to mitigation results that look strange, but given their overall results the claims are formulated with great care, and I found no evidence that they oversold the results.

Presentation

This is an essential report, so it may not be a surprise that the presentation is pretty polished. The results are clearly presented and the quality of the writing is excellent. The simulations are extensive, so in this case it is not realistic to have users rerun simulations within the browser. Additionally, the lack of time available makes it more than justifiable that they did not include such advanced features.

I would say that, given the time constraints, the presentation of this report is about as good as it gets.

Presentation: 5/5. In my opinion it is about as good as it can be, given the constraints.

Wrap-up

The overall picture looks like this:

It is a very strong effort, but it is not perfect. And given the huge impact of this project, we need to understand the areas where it is not perfect (as e.g. discussed in this review). Here are my recommendations, having done this review:

RECOMMENDATION TO READERS

A thorough piece of work *from the simulation perspective*, given the time constraints, and the results justify the claims. But take care when interpreting the mitigation results directly. And do stay tuned for updated/improved models!

RECOMMENDATION TO THE DEVELOPERS

Please check if Table 3 is indeed correct in the marked region and test the sensitivity of your less straightforward assumptions (such as asymptomatic infection spread and the uptake % of measures). And please share the code + input/output data.

Update late April:

The team has now released the source code, which you can find here:

https://github.com/mrc-ide/covid-sim

Disclaimer

To my knowledge, I have no conflict of interest with the authors of this report. I do, however, have a role in two ongoing EU projects on simulation: one of which focuses on simulating global challenges (HiDALGO), and one that focuses on verification, validation, uncertainty quantification and sensitivity analysis (VECMA). For any questions or comments please feel free to contact me on Twitter (@whydoitweet). Like all content on this blog, unless indicated otherwise any content I produced can be freely shared.

Lastly, please note that I made this review as a best effort exercise from my personal perspective. It is likely to contain a few mistakes, inaccuracies and misinterpretations, which I am happy to correct once I'm aware of them.

References

2008 paper: Halloran, M.E., Ferguson, N.M., Eubank, S., Longini, I.M., Cummings, D.A., Lewis, B., Xu, S., Fraser, C., Vullikanti, A., Germann, T.C. and Wagener, D., 2008. Modeling targeted layered containment of an influenza pandemic in the United States. Proceedings of the National Academy of Sciences, 105(12), pp.4639-4644.

PDF preprint available at: https://www.researchgate.net/publication/5520680_Modeling_targeted_layered_containment_of_an_influenza_pandemic_in_the_USA

Read more...

A short review today, as this is my last day that I am still physically working at Brunel. This simulation came to me a few days ago via Twitter:

It's a similar agent-based random movement simulation as the one presented in the Washington Post (which I reviewed previously here), and you can find the main article here:

https://medium.com/@gallaghersam95/exponential-growth-e064cc3a54ea

The simulation consists of 3000 agents moving in a 300x300 unit starting space, allowing for nodes to travel outside their original bounds. Here is another snapshot of the simulation, where you can see a random movement on the left, and one where nodes try to retain distance from each other on the right.

Source: *Medium.com*.

The Review

In the review, we cover four areas (as mentioned before) in order: Robustness, Completeness, Quality of Claim, and Presentation. I maintain this order, because the last one is pointless if the first one isn't there, and the third one is not a total disaster if the underlying model is at least of some use. So let's have a look at the

Robustness

The simulation is clearly intended as a simple proof of concept, and the author is careful not to draw overly strong conclusions from his work. However, there are two aspects that really do hamper the robustness of this simulation. First, there's no link to the source code and second, there is no explanation of the algorithm used to do the social distancing. I hope Sam Gallagher is able to add those to his Medium post soon, and I'll be happy to bump up the rating :).

Aside from that, it is a little idiosyncratic to have an initial 300x300 space, but then have agents move outside of that space. Would an initial 500x500 space with static boundaries perhaps have been more realistic, given that it's unlikely that Covid-19 patients will migrate to Greenland or the middle of the Sahara?

One thing where this simulation outperforms the previously reviewed one is in the number of agents. With 3000 agents instead of 200, it is less likely that the user would observe one-off phenomena in the results.

Robustness rating: 2/5. To some extent there is robustness in simplicity, but the lack of source code and ruleset information makes this model very hard to scrutinize.

Completeness

Again, the proof of concept scope warrants a lower bar in this aspect, but it is important to reflect on whether the things that are incorporated are also the essential elements.

For sure this model is less complete than the one I previously reviewed. There's no notion of quarantining (not so problematic, given the tight scope), but moreover there is no notion of recovery. The recovery element really is quite useful to have, as it would result in a much more realistic epidemiological curve, so perhaps that's something worth adding?

Lastly, the lack of a clearly described algorithm on the distancing in the article also affects this rating to some extent.

Just to showcase: the number of infections doesn't decrease because the simulation doesn't incorporate recovery.

Completeness rating: 2.5/5. A lot is missing, and for sure the addition of recovery would have been nice.

Quality of Claim

The author claims that “The Distancing system does a better job of reducing the spread of infection, even though the system is far from perfect. This tells us that a little bit of caution can go a long way, even just reducing some contact seems to help contain the spread.”. This is a very modest claim, and the underlying model provides reasonably good evidence for this claim.

While the model has a number of limitations, the only major limitation that could introduce bias in favour of this claim is the lack of boundaries in the bounding box. However, this is partially compensated by the fact that the box is small to start with, and that the simulation does not progress for too many steps.

Quality of claim rating: 3.5/5. There is a bit of bias, but the main claim is largely proportionate with what the underlying model showcases.

Presentation

The presentation of the model is mainly done through an animated gif. This animated gif is clear, and showcases the simulation results well overall. Being red/green color blind myself, I did not find that the dots clashed in such a way that I could not see the patterns. However, the dots are much bigger than the grid they're mapped on, which means that (a) they overlap and (b) they appear to touch each other even when they don't in the underlying simulation. My first recommendation would be to make the dots proportional to the grid. Second, it would be nice if the simulation could be interactive, instead of using a static image. Perhaps the use of Brython and/or repl.it, as I showcased in previous posts, could help there?

Lastly, it's very important to include a link to the source code, which is currently lacking.

Presentation rating: 2.5/5. It's functional and clear, but I'm deducting some points for the lack of source code, the ambiguous visualization, and the lack of interactive elements.

Wrap-up

The overall picture looks like this:

RECOMMENDATION TO THE READERS:

Good for getting a simple point across quickly now, and perhaps come back in a month to check for a better version?

RECOMMENDATION TO THE DEVELOPER:

Please do write a technical breakdown of this system, and don't give up on enhancing it just yet (!)

-> Continue to the Next Review (3)

Okay, time to get started on the simulation that gave me the idea of doing this series.

Two days ago, the Washington Post published an article with an in-browser agent-based model of a disease dubbed “simulitis”, which is meant to roughly resemble COVID-19 in simplistic terms. The article is freely available, and can be accessed here: https://www.washingtonpost.com/graphics/2020/world/corona-simulator/

(Oh, wait... there's no “simulitis” in the URL there. But don't be confused, it really is a simplified digital version of coronavirus.)

Anyway, the article showcases four simulations of a viral spread:

  • one simulation with an uninhibited spread,
  • one with a quarantined region,
  • one with a social distancing (75% of the people don't move),
  • and one with a social distancing, where 87.5% of the people don't move.

The main thrust of the article is that a) taking measures help, and that b) social distancing works better than quarantining.

Simulation screenshot of a quarantined system (early stage). Source: *Washington Post*.

At the end of the article, the authors note that the simulation is not fully realistic, because Covid-19 can kill, whereas everyone stays alive in this run.

The article was tremendously popular, even to the extent that Ex-POTUS Barack Obama tweeted about it, see:

https://twitter.com/BarackObama/status/1239267360739074048

Now, let's look at this in a bit more detail...

The Review

In the review, we cover four areas (as mentioned before) in order: Robustness, Completeness, Quality of Claim, and Presentation. I maintain this order, because the last one is pointless if the first one isn't there, and the third one is not a total disaster if the underlying model is at least of some use. So let's have a look at the

Robustness

The code is a simplistic implementation that runs in the client's browser, and has a number of assumptions. Several of those are clearly justified and documented: for instance, simulitis always spreads to other people when they touch each other so that you don't have to run for weeks to get a result. But there are a number of other assumptions that are not so explicit.

First, the quarantining run starts with a tiny gap, which changes as the run progresses. See the picture above for the starting stage, and the picture here for what it looks like later on:

Simulation screenshot of a quarantined system (late stage, quarantine area is on the left). Source: *Washington Post*.

In this image, we can see two important assumptions in this simulation: 1. the quarantine weakens as time progresses, and 2. agents are not placed back into the quarantine area after they've been classified as ill.

Now these assumptions are not explained in the article, and may be much harder to justify. In regards to (1) there may be a case for weaking quarantine conditions over time, but does that make sense when the number of infections rise? And in regards to (2) I think the whole point of quarantining is that newly ill persons are placed in it.

The other two runs showcase social distancing, and those have a more straightforward implementation that is easier to justify. One implicit assumption there is that the percentage of people doing social distancing remains the same throughout the simulation. I think this assumption is slightly questionable, but less controversial than the two I highlighted in the previous simulation.

Basically, one could reasonably expect social distancing measures to become less effective over time, as people lose the discipline to adhere to them. In that case, the percentage of stationary agents would have to decrease over time. But, of course, one could also think that discipline goes up over time. It'd be interesting how the extened of public discipline affects the accuracy of these runs.

Lastly, it is not clear to what extent the simulation has been validated against data.

Robustness rating: 3/5. The basic mechanisms broadly make sense, but the quarantine run has major issues.

Completeness

We have a basic proof-of-concept simulation in the browser, so obviously the completeness of it all is going to be limited. Many of the shortcomings and omissions are detailed in the article, so I won't count those down for completeness.

One limitation that is not mentioned, however, is the fact that only direct person-to-person transmission is incorporated, and that there are no mechanisms for transmission via surfaces.

Although person-to-person transmission is understood to be the main mechanism, there is also evidence that points to transmission via surfaces. In addition, people move around randomly, whereas real-life movement is likely to be more structured and clustered.

Other elements lacking include: (a) the possibility to have a reduced infection rate (it's 100% in the simulation, so every touch is an infection), which would make the progression somewhat more realistic, and (b) the lack of mortality, as noted by the authors already.

But overall, given the overall simplicity, it seems that many important elements have been incorporated.

Completeness rating: 3/5. No surface transmission and fully randomized movement limit the rating, but given the simplicity of the model, the design choices made in terms of incorporating elements are not unreasonable.

Quality of Claims

As a reminder, the main claims in the article are that a) taking measures help if people stick to them, and that b) social distancing works better than quarantining.

The first claim (a) is absolutely well justified, and the simulations clearly showcase how good discipline helps 'flatten the curve', irrespective of the measure taken.

The claim that social distancing works better than quarantining (b) is not sound, however, because several of the weaker assumptions in the simulation bias the whole system towards making quarantining ineffective. Because these assumptions are not at all mentioned in the article I would argue that this claim is not well justified at all.

Quality of Claim rating: 2.5/5. Two claims, of which one is clearly evidenced and one is borderline misleading. So it seems fair to give half points.

Presentation

Running simulations inside a newspaper article is a great idea, and the system works really smoothly. There are no dials or buttons to play with, but that may be for the better as that could soak up more compute resources from the user's side. The presence of different runs in different parts of the article adds even more depth to the experience, and lastly the simulations neatly pause when you scroll away.

So overall, I think this is really well-done in terms of presentation. I only deduct half a point because I find the color scheme somewhat vomit-inducing...

Presentation rating: 4.5/5. This is how you present simulations in a newspaper article!

Wrap-up

The overall picture looks like this:

RECOMMENDATION:

Play with the code, follow the advice, and forget about the rest of the article ;).

-> Continue to the Next Review (2)

It's time for a new series, given the current circumstances. I have noticed a rapid rise in online simulations of corona virus spread on the web, many of which have gone “viral” with little scrutiny.

Now I can't make vaccines, but I can scrutinize simulations. I've been modelling systems (and publishing papers about it) ranging from star clusters, to migrating people, blood flow in the brain, fluid turbulence and layered nanocomposite materials over the past two decades, and by now I believe I have a reasonably good sense of how to distinguish solid simulations from flaky ones.

So hereby I introduce a new series: the COVID-19 Simulation Review Bonanza.

The reviews are going to be both reasonably short and be intended to be constructive, with one simulation model covered in each review. My aim is to:

  1. Give the developers pointers to shortcomings, pitfalls, and ways to improve their simulation.
  2. Make the audience aware of the limitations of particular simulations, their usability, and the quality of the claims derived from it.

To do this, I'll prove a preamble describing the models, followed by a review across four criteria:

  1. Robustness: are the assumptions of the model well substantiated? Are the techniques appropriate and correctly used? Is there good scientific literature that underpins the model?
  2. Completeness: how much of the relevant phenomena are in the model? Are any obvious ones missing?
  3. Quality of Claim: here I will look at the wider context of the article presenting the model. Is the model presented appropriately, and are claims made in the article reasonable, given the quality of the underlying model? Is there good scientific literature that underpins the article itself?
  4. Presentation: is the model presented in a way that the general public can easily relate with? Is it easy to reproduce the simulations, or to scrutinize the underlying code?

At any point I am happy to make changes to this review format (I will update this post when I do so), correct reviews if simulations get updated, or do reviews on request (for free, subject to my time availability).

Just let me know via @whydoitweet when I need to do so!

I am not an expert, and I write this purely from my personal perspective, with limited knowledge. I do try to critically reflect on the information I see around me, and there is a small aspect I'd like to share about this all.

The past days a lot of big news got passed around. Stock markets are crashing, different countries are taking measures across the board: ranging from the insufficient to the extreme and justified and to the extreme and counter-productive (I'll leave it up to you to decide which measure fits in where).

In my own life, trips are cancelled left and right, and it seems a matter of time before a larger shutdown will happen.

Now one site that helped me keep a clear idea of what's going on is Worldometers, and I just want to ensure that you are accessing at as well. You can find their coronavirus page here:

https://www.worldometers.info/coronavirus/

Now a lot of media sources write about the number of cases, but we have to realize that the number of cases detected will depend on the number of tests performed. And the testing rates vary heavily by country, and countries like the US and the UK are performing much fewer tests per person than, say, South Korea or Italy. So, it's almost certain that the numbers in those countries are underestimations.

What, in my opinion, gives a slightly clearer image, is the number of deaths recorded. Assuming a steady mortality rate across different countries, the number of deaths give an indication of (a) how long the epidemic has been going on and (b) how many cases there could roughly be. And that graph, globally currently looks like this:

Source: *worldometers.info*.

And in the US it looks like this:

Source: *worldometers.info*.

I am displaying low plots, because a steady sloped line there most likely indicates that the epidemic is not curtailed.

Extrapolating lines is always a dangerous business, and I do not possess a crystal ball. However, what I do see is that the global line is sloping slightly up, and that many national lines (like the one for the US above, but also for e.g., the UK) are not flattening. So do brace for worsening situations in countries like these.

The line is clearly flattening in the case of China and South Korea though, and that gives us evidence that this pandemic can at least be fought effectively in some manner.

The coming weeks will give us a painful record of which governments are, indeed, able to do so.

What we're doing here in London at the moment

In the UK, there is no total shutdown at this stage. However, we did decide to avoid large gatherings, wash hands a lot, and avoid public transport (bus, metro) at busy times for now. In practical terms, this means a lot of commuting by bicycle for now. And we're limiting/eliminating international travel.

Let's see what the situation will be in a week from now.

Ask me 6 months ago what I thought would dominate the news in March 2020, and I would have answered the democratic primaries, brexit, or a tension between two major superpowers. All of it would be wrong of course, as the days are now dominated by the coronavirus epidemic.

I live in London myself, and no major measures have been taken yet. I do expect this to change as the number of cases is rising rapidly, and it's quite likely that we'll all face periods of self-isolation here as well.

I'll have to wait and see what this all means in concrete terms, but as a first sign in my life a workshop in Munich where I was meaning to present got cancelled last weekend.

In other news, the busy period I was talking about a few posts earlier will be coming to an end towards the end of March. I will do my best to squeeze out a few nice posts this month nevertheless, and then hopefully pick up the pace again further in April and beyond :).

Credits:

Header image courtesy of Arek Socha from Pixabay.com.

In the previous edition of GameWorld we added support for tile images, but make those images by hand is rather tedious. Today, we will begin with making your life easier, by developing a tile image generator. These images are useful for 2D games, but could possibly serve a lot of other purposes as well :).

This part will be the first of a sub-series as one can create a huge number of types of tiles. We will start with plane tiles, as well as basic grid and brick structures. However, I fully intend to move on to things like houses, trees and even sprites in due time!

Mini-sidebar: Pixel art on Coil

Now I cannot and should not continue to descend into the world of Pixel-Art related topics without paying a brief homage to AussieNinja. He provides a range of very nice pixel art tutorials, e.g. on walking animations, dungeon sceneries, your own avatar or Santa Claus. Have a look if you value the manual craft, it'll be worth it! As for me, I suppose I'm especially interested in generating stuff automatically...

Because we are just generating images, there is no need for Brython or other sophisticated game engines. Instead, we'll simply resort to using Pillow, which we used a while ago, and which is working well on repl.it again :).

Groundwork for the Tile Generator

In this tutorial we'll make two files:

  • tile_patterns.py – which contains the tile objects and a range of pattern functions we can apply.
  • main.py – which we use to generate the tileset and a range of different tiles.

I will start of with building the library in tile_patterns, and to do so we will first need to import PIL:

from PIL import Image, ImageDraw

And define an object to store the tiles in, and manipulate them.

class Tiles:

In this class, we make an init function, which creates an empty tileset with num_tiles tiles. We do this as follows:

def __init__(self, num_tiles):

self.tw = 47 # tile width (adjusted for weird pillow drawing)

self.th = 47 # tile height

self.tws = 49 # tile width with whitespace

self.ths = 49 # tile height with whitespace

self.img = Image.new('RGBA', (4*self.tws, int(int(num_tiles+3)/4)*self.ths), (0,0,0,0))

self.d = ImageDraw.Draw(self.img)

All the work is done in the last two lines, where we create a rectangular transparent image, and a drawing canvas. Everything else is just about storing values that we are going to need very often. Because we are going to define patterns, patterns of patterns, patterns of patterns of patterns etc., we are going to need very easy access to many low-level variables.

These member variables help with that, but we'll also define a few so-called getters. Here's one for getting the coordinates in the image file of a tile at place index:

def get_tile_coords(self, index):

return (index%4)*self.tws, int(index/4)*self.ths

Here's to get the x coordinate only:

def tx(self, index):

return (index%4)*self.tws

And the y coordinate:

def ty(self, index):

return int(index/4)*self.ths

Next up is a trivial function to save our tileset object as an image (I love Pillow with this):

def save(self):

self.img.save("tiles.png")

And lastly, a function that returns all the main properties of a particular tile:

def get_properties(self, i):

return self.d, self.tx(i), self.ty(i), self.tw, self.th

This includes respectively the drawing canvas, the x,y location of the corner where we start drawing, and width and height of the tile.

Okay, enough about this groundwork, let's draw ourselves a few patterns!

Drawing patterns

Because we defined such a simplistic class, drawing basic patterns is relatively simple too. Here's how we draw a plain tile of a single color:

def draw_plain(t, i, color):

d,x,y,tw,th = t.get_properties(i)

d.rectangle([x, y, x+tw, y+th], color)

...which just involves drawing a filled rectangle. For color you can put all sorts of things, but I personally prefer the short hex notation #rgb (e.g. “#FF0” for yellow), or the “rgb(255,255,0)” style notation. Remember though, it should always be a string :).

Next up, we make a function for verticle lines, each of them bw pixels apart (and a default light grey color):

def draw_vlines(t, i, color="#aaa", bw=8):

d,x,y,tw,th = t.get_properties(i)

for xi in range(0,t.tw):

if xi%bw == 0:

d.line([x+xi,y,x+xi,y+th], color)

And we do the same for horizontal lines, bh pixels apart:

def draw_hlines(t, i, color="#aaa", bh=12):

d,x,y,tw,th = t.get_properties(i)

for yi in range(0,t.th):

if yi%bh == 0:

d.line([x,y+yi,x+tw,y+yi], color)

Now, with these three functions, we can already make plain tiles, and gridded ones, by combining horizontal lines and verticle lines. Let's make a short-hand function for grids as well then:

def draw_grid(t, i, color, bw=12, bh=8):

draw_vlines(t, i, color, bw)

draw_hlines(t, i, color, bh)

I really like this one, as it's a schoolbook example of how programming can be rather simple if you break it down into pieces that are small enough :). But this grid doesn't have a background, so let's make a grid with a background too:

def draw_plaingrid(t, i, color, grid_color="#aaa", bw=12, bh=8):

draw_plain(t, i, color)

draw_grid(t, i, grid_color, bw, bh)

Lastly, just to add a little flavour to this tutorial, I'll share a basic function to make brick-like patterns, where the verticle lines vary in location:

def draw_brick(t, i, cement_color, bw, bh):

d,x,y,tw,th = t.get_properties(i)

if bw%2 > 0:

print("Error: brick width must be even.")

Here we draw he horizontal lines as normal...

draw_hlines(t, i, cement_color, bh)

But for the vertical ones, we go a little bit more exotic, with a flip-flop style branching mechanism:

yi=0

while yi < t.th:

if yi % (2*bh) == 0:

for xi in range(x,x+t.tw):

if xi%bw == 0:

d.line([xi,y+yi,xi,y+yi+bh], cement_color)

else:

for xi in range(x,x+t.tw):

if xi%bw == bw/2:

d.line([xi,y+yi,xi,y+yi+bh], cement_color)

yi += bh

Lastly, let's do bricks with a background :)

def draw_plainbrick(t, i, color, brick_color="#aaa", bw=12, bh=8):

draw_plain(t, i, color)

draw_brick(t, i, brick_color, bw, bh)

All this gives us functions to draw with, but we still have to actually draw of course!

Drawing the tiles

Now, using this library is meant to be simple. Let's see if you think that's indeed the case.

We start off with a few necessary imports:

from PIL import Image, ImageDraw

import tile_patterns as tp

And a function for our main code:

if __name__ == "__main__":

We make a tileset with 12 tiles:

t = tp.Tiles(12)

...with a transparent tile on spot 0:

...a dark green tile on spot 1:

tp.draw_plain(t, 1, "#080")

...a fine-gridded dark red tile on spot 2:

tp.draw_plaingrid(t, 2, "#800", "#aaa")

...a plain “water” tile on spot 3:

tp.draw_plain(t, 3, "#33F")

...a plain “sand” tile on spot 4:

tp.draw_plain(t, 4, "#FF4")

...a dark gray grid tile on spot 5:

tp.draw_plaingrid(t, 5, "#444", "#aaa")

...a dark gray fine brick tile on spot 6:

tp.draw_plainbrick(t, 6, "#444", "#aaa")

...a dark red coarses brick tile on spot 7:

tp.draw_plainbrick(t, 7, "#A11", "#aaa", bw=24)

...a plain rock surface tile on spot 8:

tp.draw_plain(t, 8, "#DDD")

...followed by white stone

tp.draw_plainbrick(t, 9, "#DDD", "#555", bw=24, bh=12)

...large yellow tiles

tp.draw_plaingrid(t, 10, "#DD3", "#333", bw=16, bh=16)

...and fine yellow tiles:

tp.draw_plaingrid(t, 11, "#DD3", "#333", bw=6, bh=6)

Finally, we save the whole thing and print “done” just to tell the user that the image has been generated :)

t.save()

print("done.")

Now if you run this program, it will generate a file called tiles.png. And that file will look like this:

Closing thoughts

Now this tile generator is not nearly enough to make a whole game with, but at least you can do a few basic terrain tiles, floors and walls with it.

Lastly, you can find the Repl for the Tile Generator here:

Feel free to fork it and make it suit your purposes :).

For subscribers I'll briefly share what will come up next in this area in future editions:

Read more...

We make models every day. And if we were robots, we'd be making simulations every day. This post is about making you (more) aware that you build models yourself every day (not only when you follow my dreary/delightful tutorials), and to help you a little bit to scrutinize them.

Climate change is real, and the world is roughly spherical. These are facts, because there is a huge amount of rigorously reviewed literature confirming it, as well as a huge amount of high-quality observational evidence. But most of the other things we assume to be true are not so well substantiated. And those I refer to as assumptions. They are things that we believe in, or take for granted, but where the evidence isn't always as rock-solid. For example:

  • “Trump will win the next election.” or “Trump won't win the next election.”
  • “It's not good to leave a window open when the heating is on.”
  • “I cannot pass this exam unless I study for an hour every day, starting today.”
  • “Milk is good for me.”
  • “My friend is being boring because she/he is tired.”

Now let me make a small side-track for those interested, before I proceed with the main content on appraising your assumptions.

Side-Track: facts and assumptions in science & simulation

Some disciplines in science find themselves in a position of luxury. They can base their works on physical laws, which have been tested and proven right for countless decades. Laws such as Newton's Law of Gravity for stellar dynamics, or the Navier-Stokes equation in fluid dynamics, can be treated as facts in many situations, and provide a solid foundation for those who want to do science and build simulations in these fields.

In other disciplines, such as many studies of human behavior, we do not possess these luxuries, and we have little choice but to resort to assumptions. Especially in those cases, for instance when I was working on a new rule set for a migration simulation this week, we need to be able to distinguish the stronger assumptions from the weaker ones. And that last thing is exactly why I felt about making this blog post.

Back to the main track: why do we need assumptions?

So first of all, we need to have our mental models to make decisions, and almost of all of these models will need assumptions, because:

  • The facts alone do not provide enough information to make a decision.
  • We are not aware of all the facts in existence.
  • ..and even if we were, often our mental image of a given fact can be subtly distorted compared to the raw fact.

Let me give you an example of the third one:

Image courtesy of *Tumisu at pixabay.com*.

It's universally established that the planet is undergoing climate change, and that the temperature is increasing. Most (unfortunately not all) of us have internalized that, and have a mental image of what that means. However, the interpretation of the details of climate change and its effects vary per person. For instance, you may have assumptions about the expected:

  • Temperature increase: we have 1.5 degrees now I believe, but do you expect 2 degrees, 3, 4 or 5 degrees in 2050? The expectation is likely to be colored by your assumptions about society as a whole.
  • The increase in seawater level: yes, global climate change is happening, but what would the sea level be in 2050? Again, your assumptions about society will color this opinion, but it may also be affected by the partial, yet not conclusive, evidence that has been provided by researchers or journalists.
  • How all this affects food, people and peace on the world: it seems obvious that a change is afoot, but any exact and firm expectation about the change is likely to rest on assumptions once more, and be an assumption of itself.

Without these assumptions, we struggle to estimate the concrete effects of climate change. And in general, assumptions are the glue we use, and need, to make decisions.

What matters with assumptions?

And that's all fine, but we can't rest on our laurels, for two reasons:

One, because assumptions aren't as solid as facts, there is a big risk that we are wrong, or that we will be wrong in the future. The worse the justification of our assumption is, the more likely to happen.

Two, some assumptions are just minor figments of our mind (indeed, some may only appear in a dream), but we may use other assumptions to make important decisions, or to spread them to other people if we are firmly convinced and find them rather important.

To keep matters very simple at first, we therefore end up with this appraisal spectrum of assumptions:

The diagonal arrow represents the ideal arrangement for assumptions. We'd like the better-justified assumptions to be more important, and the ill-justified ones to be irrelevant. After all, the assumptions in the bottom right may end up as trivia, because they are unimportant yet well-justified. The ones in the top-left are the ones to watch for, as they are highly important but poorly justified. In particularly, these could create problems for yourself and your credibility, as well as to others and society at large.

But to make that call whether your assumptions are roughly on the diagonal arrow, and indeed map any assumption to this diagram, you'll first need to appraise the assumption :).

Now making a detailed appraisal scheme is beyond my expertise, and would make this blog post enormous in any case. But I am happy to provide a few (hopefully useful) general pointers.

0: What are your assumptions?

Are you really aware of what your own assumptions are? There are probably a near-infinite number, but how about you try to write down a couple (say, 5 to 20 or so) that you have on your mind now?

1: How important is your assumption?

In simulation-building we can quantify this relatively easily, namely by doing something called Sensitivity Analysis, where we estimate the effects of changing one or more assumptions on the outputs of our model.

Though it'll be hard to produce exact numbers in your own context, doing a “sensitivity analysis” on your own assumptions can be very helpful. In essence it's asking yourself what would change in your life, and around you, if you were to invert the assumption that you're appraising?

When thinking of what would/could change, you may want to keep in mind the following possibilities:

  • Did your assumption contribute to any decisions you made? And how important where those decisions?
  • If you were to find someone in a conversation who disagrees with your assumption, would you try to convince that person of your assumption?
  • Did you go out of your way to convince people of your assumption without being prompted, either on the internet or in real life?
  • Is the assumption necessary to justify the existence of one or more of your other major assumptions?

I think you can appreciate that an assumption will be more important if you tend to find more evidence that falls within the scope of the four points above :). And perhaps that allows you to rank your assumptions in importance?

2: How good is the justification of your assumption?

To put your assumption on the horizontal axis of the diagram above, it's important to appraise how well you can justify it. However, this can be a bit harder to assess.

There is a lot of literature about quantifying the level of justifications, and in this post I will scratch the surface of it at the very most. I thought for a whole how I could make something that is likely to be reasonably on target, and relatively easy to consult. So, I hereby present the Assumption Quality Checklist Infographic:

Using it is pretty simple. The more green items you find vs. red items (and the more important the green items are vs. the red), the further you can move your assumption to the right on the diagram above :). As for the Tie-breakers, simply treat a “Yes” answer as a Quality Booster and a “No” answer as a Quality Buster :).

3: Putting the assumption on the diagram

Once you've gone through those steps, you can put the assumption on the diagram. And you can do the same with all the others that you want to appraise.

What kind of patterns emerges for you? Do you get a diagonal line, or are many assumptions actually in the top left and bottom right corner?

If any of your assumptions end up far above the diagonal arrow, you may want to either reconsider their importance, or try to see if there's any way you can justify it better.

And if they end up well below the arrow... well, I guess you identified some trivia then ;).

Closing thoughts

There may well be material around online that covers this better than I did. But honestly, I had trouble finding it!

I am very curious to learn (you can reach me at @whydoitweet on Twitter) how easy you found it to map your assumptions to the appraisal spectrum. And did you get the patterns you expected? And did you find it useful?

For subscribers I provide a few interesting links related to all this below.

Credits

Header image courtesy of pxhere.com (CC-0).

Read more...