School Streets
Will closing London's school roads actually reduce air pollution?
In 2020 The Greater London Authority commissioned a study to determine the extent to which local air pollution levels can be reduced by closing school roads.
The recently published report, based upon a sophisticated monitoring campaign, calculates the reductions in air pollution exposure that school children will benefit from.
These calculated benefits have subsequently been cited in various press releases surrounding the study, including quotations from the Mayor himself.
For example, Sadiq Khan is quoted in the FIA Foundation blog as saying:
‘It’s great to see the huge reduction in nitrogen dioxide during pick up and drop off on schools streets – a time where countless children and adults would otherwise be exposed to dangerous emissions.’
As I show here, there is a strong argument to say that it wasn’t possible to identify any conclusive reduction in pollutant levels, despite the large amount of data generated, let alone a ‘huge reduction’.
This newsletter takes a closer look at the data, and highlights a few surprising admissions along the way.
Background
The Mayor’s Streetspace for London plan was initiated in response to the coronavirus pandemic, a component of which was the ‘School Streets’ initiative. Roads were closed during drop-off and pick-up times outside a number of schools in the Lambeth, Enfield and Brent boroughs of London to aid with social distancing.
During the road closure periods of 2020, the GLA and its partners - Bloomberg Philanthropy and the FIA Foundation - took the opportunity to commission a team of air quality experts to monitor local air pollution levels.
A total of 30 monitoring locations were selected in the vicinity of a number of schools. Monitoring equipment was deployed to continuously measure nitrous oxide (NO) and nitrogen dioxide (NO2).
To get a quick primer on air pollution science, you may find this earlier newsletter useful.
Claims
The report concludes that pollution reductions of up to 34% were seen outside of the schools due to the closures.
Wait, what?!
I have almost two decades of air pollution experience behind me, some of which was working in and around London. I am aware of the limitations of this type of study, and I was curious to see how they were addressed. Hence why I decided to take a closer look.
The three main problems are described below.
1. Seeing the wood from the trees
Pollutant levels are continuously changing in towns and cities. If you hold an air pollution instrument and look at the display, it will change from second to second. If you walk down the street, it will change with each step. In urban environments, the movement of people and vehicles disturb the air. Sources of pollution are rarely consistent and continuous. Then of course there is the wind, ever changing. And so on…
Moreover, the pollution monitoring instruments are highly sensitive, necessarily so in order to detect the very low concentrations of pollutant molecules in the air.
Given the highly variable nature of local pollution, it is very difficult to ascribe air pollution sources, and therefore interventions aimed at lowering pollution, especially so in urban environments. The School Streets GLA study fully recognises this issue and quotes DEFRA’s Air Quality Expert Group accordingly:
‘…interventions rarely occur in isolation from other changes that affect air quality… Indeed, not every intervention is detectable in terms of quantifying changes in pollutant concentrations or health outcomes, even using sophisticated analysis techniques.’
2. Regional vs local effects
In urban environments, pollution levels are often dominated by the compounded effects of multiple sources aggregated over time and distance.
The School Streets study makes the following statements (emphasis added):
‘‘When viewing data from all 30 locations, the level of similarity in the concentration trends is very high, suggesting that much of the NOx pollution recorded is *regional* in nature..’’ (para 2.18)
and
‘‘..the pattern of concentrations across all sites is generally very similar, suggesting that the measured levels are dominated by *regional*, rather than local effects.’’ (para 4.1)
and
‘‘This further emphasises that during the three months of the study it is *regional* emissions that are the principal driver of concentrations, rather than traffic emissions from the local roads adjacent to the monitors.’’ (para 4.12)
These are critical statements in the context of the School Street study, which is concerned exclusively with local sources of air pollution!
3. The ‘No Baseline’ problem
The study was rolled out so as to catch the period during which the school streets initiative was in place. However, no ‘before’ or ‘after’ period has been included, which represents an obvious problem.
How do you know, therefore, that the measures are making any difference? This isn’t an easy question to answer.
The report acknowledges this weakness:
‘‘There were limitations to the data collected in this study. Most notably the lack of baseline data and significant fluctuation in London-wide traffic volumes.’’ (para 4.3)
Given this inability to make direct ‘before’ and ‘after’ conditions, the study has used two alternative methods in order to attempt to determine the air pollution benefits of the road closures.
Looking at the shape of the ‘diurnal profiles’ (patterns throughout the day) to see whether any anomalies arise during the times when road closures were in place, and
Comparing the diurnal profiles of pairs of similarly located sites, where one would be expected to be impacted by street closures but not the other
Diurnal profile shapes
A ‘diurnal profile’ is the pattern changing pollutant concentrations throughout a day. Each time stamp is an average of all of the same time stamps across all days of the study period when shools were open. So for example, an average of all 00.00am recorded concentrations is plotted on the profile, the same for 00:15am and so on until you plot a complete 24hr period.
With thirty monitoring sites, a road closure happening twice a day (one in the morning, one in the afternoon), and with two pollutants being monitored, this gives 120 profiles to examine. If you subtract out of this the control sites (that could not be influenced by the school street closures) you are still left with more than a hundred opportunities to witness pollutant changes in the profiles. The ‘best’ of the observations are as follows:
3 sites (at a school in Enfield) appeared to have a dip in concentrations of NO during the morning closures.
3 sites (Sites 20, 21 and 22 outside Van Gogh School in Lambeth) at a school in Lambeth appeared to have a dip in NO concentrations during the morning closures.
Out of 100+ opportunities, only 6 pollution profiles showed anomalies that could *potentially* be ascribed to local road closures.
Moreover,
it is difficult to attribute these ‘dips’ to school street closures since fluctuations of similar magnitudes appear throughout the data when no street closures are in place,
in some case it is difficult to tell whether the anomalies were actually ‘dips’ or an artifact of elevations prior to and after the apparent ‘dip’, and
at three of the sites with the most pronounced ‘dips’ in the data, similar fluctuations also occurred at those same sites during half-term holidays, when school closure effects are irrelevant.
Quantifying the, err, benefits!
In spite of these significant uncertainties as to whether street closures have any benefits to local air pollution levels, the study, quite astonishingly (and after fully acknowledging these limitations I might add), then goes on to quantify the benefits!
The following sentence highlights the arbitrary quantification method used. Notice the use of the word ‘expected’. Ideally you would be plotting sites where influence WAS observed, not EXPECTED to be observed:
‘‘…concentrations at Sites where a greater School Street influence might be expected have been plotted against those where the effect would be expected to be smaller.’’ (para 5.1)
Conclusions
We have established that;
it is extremely difficult to ascribe local sources to observed levels of pollution, especially in urban environments, a point well understood by government departments, academics and and consultants alike,
there is no ‘baseline’ (i.e. normal conditions) against which to compare pollutant levels recorded during the street closure period,
regional influences are likely to dominate over local pollution sources,
among the 100+ opportunities to identify any anomalies in the daily pollution patterns that can be linked to road closures, none were conclusive, and
Notwithstanding the problems stated above, the assumed benefits were quantified, claiming an apparent 34% and 23% reductions in NO and NO2 concentrations respectively.
I will end by addressing the study’s stated objectives:
‘‘…to identify simple, sharable messages that can be easily communicated on the air quality benefits associated with School Streets, so as to support a case for potentially making School Streets permanent.” (para 1.4)
The objectives assume that the there is an air pollution benefit to the School Streets initiative. Accordingly, the study seeks to ‘identify’ data with which to communicate these presupposed ‘benefits’.
In other words, the study seeks to find data with which to support the presupposed outcome.
This ‘assumed effectiveness’ approach seems to be replacing evidence-based scientific and epidemiological study more and more.
This is not science.
This is Public Relations, disguised - quite well I might add - as science.
I hope you found this interesting!
-Tristan