Lindsey Pike – IEUREKA!

COVID19 – should schools close early for Christmas?

Posted on December 15, 2020February 11, 2025 by lindsey.pike

Sarah Lewis, Marcus Munafo and George Davey Smith

We have previously written about the limited risk posed to pupils, teachers and the community by schools being open during the Covid19 pandemic. Schools have now been open for almost a full academic term (3 months), so it is time to take another look at the evidence. (more…)

We should be cautious about associations of patient characteristics with COVID-19 outcomes that are identified in hospitalised patients.

Posted on June 3, 2020 by lindsey.pike

Gareth J Griffith, Gibran Hemani, Annie Herbert, Giulia Mancano, Tim Morris, Lindsey Pike, Gemma C Sharp, Matt Tudball, Kate Tilling and Jonathan A C Sterne, together with the authors of a preprint on collider bias in COVID-19 studies.

All authors are members of the MRC Integrative Epidemiology Unit at the University of Bristol. Jonathan Sterne is Director of Health Data Research UK South West

Among successful actors, being physically attractive is inversely related to being a good actor. Among American college students, being academically gifted is inversely related to being good at sport.

Among people who have had a heart attack, smokers have better subsequent health than non-smokers. And among low birthweight infants, those whose mothers smoked during pregnancy are less likely to die than those whose mothers did not smoke.

These relationships are not likely to reflect cause and effect in the general population: smoking during pregnancy does not improve the health of low birthweight infants. Instead, they arise from a phenomenon called ‘selection bias’, or ‘collider bias’.

Understanding selection bias

Selection bias occurs when two characteristics influence whether a person is included in a group for which we analyse data. Suppose that two characteristics (for example, physical attractiveness and acting talent) are unrelated in the population but that each causes selection into the group (for example, people who have a successful Hollywood acting career). Among individuals with a successful acting career we will usually find that physical attractiveness will be negatively associated with acting talent: individuals who are more physically attractive will be less talented actors (Figure 1). Selection bias arises if we try to infer a cause-effect relationship between these two characteristics in the selected group. The term ‘collider bias’ refers to the two arrows indicating cause and effect that ‘collide’ at the effect (being a successful actor).

Figure 1: Selection effects exerted on successful Hollywood actors. Green boxes highlight characteristics that influence selection. Yellow boxes indicate the variable selected upon. Arrows indicate causal relationships: the dotted line indicates a non-causal induced relationship that arises because of selection bias.

Figure 2 below explains this phenomenon. Each point represents a hypothetical person, with their level of physical attractiveness plotted against their level of acting talent. In the general population (all data points) an individual’s attractiveness tells us nothing about their acting ability – the two characteristics are unrelated. The red data points represent successful Hollywood actors, who tend to be more physically attractive and to be more talented actors. The blue data points represent other people in the population. Among successful actors the two characteristics are strongly negatively associated (green line), solely because of the selection process. The direction of the bias (whether it is towards a positive or negative association) depends on the direction of the selection processes. If they act in the same direction (both positive or both negative) the bias will usually be towards a negative association. If they act in opposite directions the bias will usually be towards a positive association.

Figure 2: The effect of sample selection on the relationship between attractiveness and acting talent. The green line depicts the negative association seen in successful actors.

Why is selection bias important for COVID-19 research?

In health research, selection processes may be less well understood, and we are often unable to observe the unselected group. For example, many studies of COVID-19 have been restricted to hospitalised patients, because it was not possible to identify all symptomatic patients, and testing was not widely available in the early phase of the pandemic. Selection bias can seriously distort relationships of risk factors for hospitalisation with COVID-19 outcomes such as requiring invasive ventilation, or mortality.

Figure 3 shows how selection bias can distort risk factor associations in hospitalised patients. We want to know the causal effect of smoking on risk of death due to COVID-19, and the data available to us is on patients hospitalised with COVID-19. Associations between all pairs of factors that influence hospitalisation will be distorted in hospitalised patients. For example, if smoking and frailty each make an individual more likely to be hospitalised with COVID-19 (either because they influence infection with SARS-CoV-2 or because they influence COVID-19 disease severity), then their association in hospitalised patients will usually be more negative than in the whole population. Unless we control for all causes of hospitalisation, our estimate of the effect of any individual risk factor on COVID-19 mortality will be biased. For example, it would be unsurprising that within hospitalised patients with COVID-19 we observe that smokers have better health than non-smokers because they are likely to be younger and less frail, and therefore less likely to die after hospitalisation. But that finding may not reflect a protective effect of smoking on COVID-19 mortality in the whole population.

Figure 3: Selection effects on hospitalisation with COVID-19. Box colours are as in Figure 1. Blue boxes represent outcomes. Arrows indicate causal relationships, the dotted line indicates a non-causal induced relationship that arises because of selection bias.

Selection bias may also be a problem in studies based on data from participants who volunteer to download and use COVID-19 symptom reporting apps. People with COVID-19 symptoms are more likely to use the app, and so are people with other characteristics (younger people, people who own a smartphone, and those to whom the app is promoted on social media). Risk factor associations within app users may therefore not generalise to the wider population.

What can be done?

Findings from COVID-19 studies conducted in selected groups should be interpreted with great caution unless selection bias has been explicitly addressed. Two ways to do so are readily available. The preferred approach uses representative data collection for the whole population to weight the sample and adjust for the selection bias. In absence of data on the whole population, researchers should conduct sensitivity analyses that adjust their findings based on a range of assumptions about the selection effects. A series of resources providing further reading, and tools allowing researchers to investigate plausible selection effects are provided below.

For further information please contact Gareth Griffith (g.griffith@bristol.ac.uk) or Jonathan Sterne (jonathan.sterne@bristol.ac.uk).

How can researchers engage with policy?

Posted on July 22, 2019July 29, 2019 by lindsey.pike

Dr Alisha Davies

Dr Laura Howe

Prof Debbie Lawlor

Dr Lindsey Pike

Follow Alisha, Laura, Debbie and Lindsey on Twitter

Policy engagement is becoming more of a priority in academic life, as emphasis shifts from focusing purely on academic outputs to creating impact from research. Research impact is defined by UKRI as ‘the demonstrable contribution that excellent research makes to society and the economy’.

On 25 June 2019 the IEU held its first Engagers’ Lunch event, which focused on policy engagement. Joined by Dr Alisha Davies, Head of Research from Public Health Wales, Dr Laura Howe, Professor Debbie Lawlor and Dr Lindsey Pike from the IEU facilitated discussion drawing on their experiences – from both sides of the table – of connecting research and policy. Below we summarise advice from our speakers about engaging with policy.

The benefits of engaging with policy & how to do it

As an academic you need to consider what your ‘offer’ is. What expertise do you bring? This may be topic specific knowledge or relate to strong academic skills such as critical approaches to complex challenges, novel methods in evaluation, health economics. Recognise where you add value; the remit of academia is to develop robust evidence in response to complex and challenging questions using reliable methods – a gap that those in practice and /or policy cannot fill alone.
Find the right people to engage with – who are the decision makers in your area of research? Listen to what is currently important to inform action / policy. Read through local and national strategies in your topic of expertise to understand the wider landscape and where your work might inform, or where you might be able to address some of those key gaps. Academics can also submit evidence to policy (colleagues from the University of Bristol can access PolicyBristol’s policy scan, which lists current opportunities to engage).
Be visible and actively engage. Find out what local events are going on in your area related to your research and go along to meet local public health professionals. It’s a good way to meet people, find commonalities and form collaborations.
Condense your new research into a short briefing, identify what it adds to the existing evidence base, how does it inform given the wider context.
As an academic you will have a network of other research colleagues. Policymakers value being able to draw on this network for information. When providing evidence, don’t just cite your own – objectivity is one of the key advantages of working with academics, and policymakers value your intellectual independence. Your knowledge of the broader evidence base is invaluable.
Setting up a research steering group or stakeholder panel can be a great way to develop your relationships and ensure your research is speaking to policy, practice or industry priorities. Key to this is getting the right people involved – this blog post from Fast Track Impact has some useful advice.

The challenges of engaging with policy & how to navigate them

Academic and policymaking timescales are different. Policymakers need an answer yesterday while academics may not feel comfortable with providing a definitive response without time for reflection. There’s a need for flexibility on both sides.
There are also tensions between the perceived need for certainty and ability to be able to provide it. Policymakers may want ‘an answer’, but the evidence base may not be robust enough to give one. It is more useful to outline what we do and do not know, with a ‘balance of probabilities’ recommendation, than to say ‘more research is needed’.
Language can also be a barrier. Academic language is complex and, at times, impenetrable; policymaker documents need to be aimed at an intelligent lay audience, without jargon, and focusing on what matters to them (outlining policy options and the evidence base behind them – not lengthy discussions of statistical methods). Look at Public Health Wales’ publications, for example on digital technology, adverse childhood experiences and resilience, or mass unemployment events, or examples from the NIHR Dissemination Centre or PolicyBristol to get a sense of the language to use.
Do you think you have time for networking with non-academic stakeholders? The perception of opportunity costs can be another barrier for academics. While time for networking might not be costed into your grant funding, think of it in the same way as writing a grant application; you can’t guarantee the outcome but the potential reward is significant.
There are no guarantees in policy engagement work, and a level of realism is required around what findings from one study can achieve. Policymaking is a complex and messy process; the evidence base is just one factor in decision making. Your recommendations may not be taken up because of politics, resource issues, or other concerns taking priority. Sometimes your relationships will reach honourable dead ends, where you realise that interests, capacity or timescales are not as aligned as you thought. Knowing this before you start is important to avoid feeling disillusioned.

Cartoon showing complexity of policymaking process and comparing it to making sausages — Policymaking is a complex and messy process; the evidence base is just one factor in decision making. Image from Sausages, evidence and policymaking: The role of universities in a post-truth world, Policy Institute at Kings 2017

In summary, the panel concluded that policymakers are interested in academic research as long as their priorities are addressed. While outcomes are not guaranteed, our colleagues at PolicyBristol advise a strategy of ‘engineered serendipity’ – looking for and capitalising on opportunities, being ready to talk about your research in a clear and policy orientated way (why does your research matter and what are the key recommendations?) and aim to build long term and trusting relationships with policymakers.

If you’d be interested in attending a future Engagers’ Lunch, please contact Lindsey Pike.

Further information & resources

PolicyBristol aims to enhance the influence and impact of research from across the University of Bristol on policy and practice at the local, national and international level.

Public Health Wales Research and Evaluation work collaboratively across Public Health Wales and with external academic and partner organisations, and are keen to facilitate research links across Public Health Wales with new national and international partners.

Research impact at the UK Parliament ‘Everything you need to know to engage with UK Parliament as a researcher’

Parliamentary research services across the legislatures include:

House of Commons Library: an independent research and information unit. It provides impartial information for Members of Parliament of all parties and their staff.
Parliamentary Office of Science and Technology: Parliament’s in-house source of independent, balanced and accessible analysis of public-policy issues related to science and technology.
Research and Information Service (RaISe): aims to meet the information needs of the Northern Ireland Assembly Members, their staff and the secretariat in an impartial, objective, timely and non-partisan manner.
Scottish Parliament Information Centre (SPICe): the internal parliamentary research service for Members of the Scottish Parliament.
Senedd Research: an expert, impartial and confidential research and information service designed to meet the needs of Wales’ National Assembly Members and their staff.

Tag: Lindsey Pike