Underestimation of Drug Use: A Perennial Problem with Implications for Policy

by Olivia Maynard

Follow Olivia on Twitter

Photo by Louie Castro-Garcia on Unsplash

In a paper recently published in the journal Addiction, Hannah Charles and colleagues suggest that the prevalence of illicit drug use among 23-25 year olds in a Bristol-based birth cohort (ALSPAC) is over twice that reported in the Crime Survey for England and Wales (CSEW). The team propose that these figures reflect under-reporting in the CSEW, although they note that they may reflect higher levels of illicit drug use in Bristol. Here I present some preliminary data supporting their view that the CSEW underestimates illicit drug use.

In March 2020, I recruited 683 UK university students to participate in a short survey on drug use via the online survey platform Prolific which has been shown to produce reliable data. I recruited only students aged 18 to 24 years who reported using alcohol in the past 30 days, and participants reported whether they had used any of MDMA/ecstasy, cocaine or cannabis in the past two years.

Table 1. Prevalence of self-reported illicit drug use across three surveys of young people in the UK

University
students
via ProlificAged 18-24
Bristol, ALSPAC

Aged 23-25

CSEW

Aged 23-25

2 years 1 year Lifetime 1 year Lifetime
Any illicit drug usea 52.7 (360) 36.7 62.8 16.4 40.6
Cannabis 50.2 (343) 29.2 60.5 13.8 37.3
MDMA/ecstasy/amphetaminesb 23.3 (159) 17.0 32.9 3.6 11.1
Cocaine 21.1 (144) 19.6 30.8 4.8 13.9

Notes: Values represent percentage of participants (number of participants). Percentages for CSEW and ALSPAC are taken from Charles et al (1) and are weighted percentages.
a ‘Any illicit drug use’ refers only to the illicit drugs assessed in the respective surveys (only cannabis, MDMA and cocaine in our survey), more drugs in ALSPAC and CSEW – see Charles et al (1).
Our Prolific survey asked about ‘MDMA / ecstasy’ use, ALSPAC categorised ecstasy/MDMA use along with other ‘amphetamine’ use and CSEW asked about ‘ecstasy’ use.

Over half of my sample reported using at least one of cannabis, cocaine or MDMA in the past two years (Table 1). This is markedly higher than the CSEW’s estimates of either past year or lifetime use, and more in line with those reported in ALSPAC. Comparing across drugs, past two-year use of the three drugs is higher in my survey than either past year or lifetime use in the CSEW, and higher than past year, but lower than lifetime use in ALSPAC. Perhaps of more interest than ever use of the drugs over the past two years, I also examined the combinations of drugs students in my survey were using (Table 2). I find that the majority of students who report using illicit drugs have only used cannabis in the past two years (25% of all students), although the second largest group (15%) have used all three of cannabis, MDMA and cocaine.

Table 2. Prevalence of self-reported illicit drug among UK university students

Qualtrics survey of university students (past two years)
All
(n=683)
Female
(n=336)
Male
(n=312)
Other
(n=35)
Illicit drug use 
Cannabis 50.2 (343) 48.5 (163) 53.5 (167) 37.1 (13)
MDMA / ecstasy 23.3 (159) 19.3 (65) 29.2 (91) 8.6 (3)
Cocaine 21.1 (144) 17.6 (59) 26 (81) 11.4 (4)
Illicit drug use profiles
Alcohol only (no illicit drug use) 47.3 (323) 48.2 (162) 44.6 (139) 62.9 (22)
Any illicit drug usea 52.7 (360) 51.8 (174) 55.4 (173) 37.1 (13)
Cannabis only 24.5 (167) 27.4 (92) 21.5 (67) 22.9 (8)
Cannabis + Cocaine + MDMA 15.4 (105) 11.3 (38) 20.8 (65) 5.7 (2)
Cannabis + MDMA 6.3 (43) 6 (20) 7.1 (22) 2.9 (1)
Cannabis + Cocaine 4.1 (28) 3.9 (13) 4.2 (13) 5.7 (2)
Cocaine only 0.9 (6) 1.2 (4) 0.6 (2) 0 (0)
MDMA only 0.9 (6) 0.9 (3) 1 (3) 0 (0)
Cocaine + MDMA 0.7 (5) 1.2 (4) 0.3 (1) 0 (0)

Notes: Values represent percentage of participants (number of participants).
‘Illicit drug use’ refers to participants reporting any use of the three drugs in the past two years.
‘Illicit drug use profiles’ refers to the combinations of drugs participants report using in the past two years.
a ‘Any illicit drug use’ refers only to use of cannabis, MDMA and cocaine.

There are some important differences between my sample and both the CSEW and ALSPAC samples. Some differences may mean that my figures are overestimates, including sampling university students who are more affluent than the general population (although drug use is not necessarily higher among students than non-students) and only including those who reported drinking alcohol (although according to the study authors, over 95% of the ALSPAC participants report past year drinking). Other differences may mean my figures are underestimates, including only asking about use of three drugs (thereby underestimating ‘any illicit drug use’), and the younger average age of my sample. I also report on past two-year use, rather than either lifetime or past year use as per CSEW and ALSPAC. Given these differences, I would like to run a larger, more representative sample on the Prolific platform (Prolific allows researchers to recruit a sample which is representative of the general population), to get an estimate of illicit drug use which is more comparable to ALSPAC and CSEW.

Despite these differences, my data support those reported by Charles and colleagues. Indeed, I find it unsurprising that illicit drug use is under-reported in the Home Office’s CSEW. The validity of self-reports for sensitive issues has long been a concern. Over-reporting of illicit drug use is not considered to be a concern and numerous methods have been developed for preventing under-reporting (see a 1997 NIDA report on this issue, as well as more recent techniques for estimating prevalence of use such as the crosswise method). It is important to consider the context in which surveys are administered, including participants’ perception of who is asking the questions and for what reason. It seems that if drug use is asked about in a research context, (e.g., with a clear research objective, informed consent and no possibility of repercussions), the validity of responses may be higher than when questions are asked by organisations that are perceived to be involved in the punishment of people who use drugs (e.g., governments, universities).

While the CSEW recognises that it does not reliably measure problematic drug use, my data and that of Charles and colleagues provide evidence that CSEW’s claim that it is a ‘good measure of recreational drug use’ may be wrong. Although it may be convenient to believe that only a small subset of the population uses illicit drugs, accurate information may galvanise policy makers (both nationally and locally, including at universities) into developing drugs policies that reflect reality and which support, rather than criminalise, the large proportion of the population who choose to use drugs. Indeed, this is what we’re doing at the University of Bristol, where it has been accepted that drug use is relatively common among our students and we’re providing support and education to those students who need it.

 

This blog posted was originally posted on the Tobacco and Alcohol Research Group blog

Using evidence to advise public health decision makers: an insider’s view

This blog post reviews a recent seminar hosted by the MRC IEU, PolicyBristol and the Bristol Population Health Science Institute.

Public health is one of the most contested policy areas. It brings together ethical and political issues and evidence on what works, and affects us all as citizens.

Researchers produce evidence and decision-makers receive advice – but how does evidence become advice and who are the players who take research findings and present advice to politicians and budget-holders?

We were pleased to welcome a diverse audience of around 75 multidisciplinary academics, policymakers and practitioners to hear our seminar speakers give a range of insider perspectives on linking academic research with national and local decisions on what to choose, fund and implement.

In this blog post we summarise the seminar, including links to the slides and event recording.

Seminar audience
Seminar attendees in the Coutts lecture theatre. Image credit: Julio Hermosilla Elgueta

Chair David Buck from The King’s Fund opened the event, highlighting the importance of conversations between different sectors of the evidence landscape, and of local decision-making in this context.

‘The art of giving advice’

The session was kicked off by Richard Gleave, Deputy Chief Executive, Public Health England, who is also undertaking a PhD on how evidence is used in public health policy decision making.

His presentation ‘Crossing boundaries – undertaking knowledge informed public health’ set the context, observing that most academic teams – from microbiology labs to mental health researchers – aim to improve policy and practice; but ‘the art of giving advice is as important and challenging as the skill required to review the evidence’.

Richard then introduced a range of provocations and stereotypes about how the policy decision making process can be framed.

Citing Dr Kathryn Oliver, he encouraged attendees to challenge the idea that there’s an ‘evidence gap’ to be crossed, and instead focus on doing good working together to improve the public’s health.

Giving an example of the Institute for Government’s analysis of how the smoking ban was enacted, he noted the role of a small number of influential groups and individuals in securing a total ban in 2007. He encouraged actively crossing the boundaries between academia, policy and practice, and working with boundary organisations and influencers as part of this process.

‘Partnerships between science and society’

Professor Isabel Oliver gave a second national perspective.

Speaking as a research-active Director of Research, Translation and Innovation and Deputy Director of the National Infection Service, she suggested that ‘Partnerships between Science and Society’ are the key to evidence based public health.

She questioned why is it when we have such an abundance of research, we still don’t have the evidence we need? And why does it take so long to implement research findings? She argued that a key issue here is relevance; how relevant is the research being produced, especially to current policy priorities?

Isabel outlined challenges including:

  • Needing evidence quickly in response to public health emergencies, and not being able to access it, for example how to bottle-feed babies during flooding crises, or whether to close schools during flu pandemics
  • Mismatched policy and research priorities; e.g. policy needing evidence on the impact of advertising on childhood obesity, but research focusing on the genetics of obesity
  • The (unhelpful) prevalence of ‘more research needed’ as a conclusion, and knowing when the evidence is sufficient to make a decision
  • A need to develop trust between stakeholders, made more challenging by the frequency of policy colleagues moving roles.

She also questioned whether the paradigm of evidence-based medicine works for complex issues such as public health or environmental policy.

Isabel concluded with some observations; that broader and more collaborative research questions that address the real issues are needed; and collaborating with a broad range of stakeholders, including industry and finance, should not be discounted.

She finished by reiterating a call for public health advice that is relevant, and responds to a policy ‘window’ being open.

Seminar speakers L-R: Dr Olivia Maynard, Richard Gleave, Professor Isabel Oliver and Christina Gray. Image credit: Julio Hermosilla Elgueta

‘Local perspective’

Christina Gray, Director of Public Health at Bristol City Council gave the local view, providing a helpful explanation of her role and the process of decision making within a local authority.

She outlined three key principles:

  • The democratic principle; elected members are ‘the council’; officers (including her role) provide advice. Local authorities are close to their people and are publicly accountable. Their decisions are formally scrutinised and need to be justified, and resource allocation is a key – and stark – challenge, especially in the context of austerity.
  • The narrative principle: how the society that the authority represents holds multiple legitimate (and competing or conflicting) perspectives and realities, which all need to be considered.
  • The (social) scientific principle; the development of human knowledge in a systematic way – which is then shared into the democratic process, as one of a range of narratives.

Christina outlined a case study example of an initiative on period dignity which Bristol City Council is leading as part of Bristol’s One City Plan, and how the evidence base for the programme was located and used. She posed the question of what evidence matters locally, and suggested that evidence of impact, economic evidence, and retrospective evidence that demonstrates whether what has been done works, in order to build on it, are the most helpful. To close, Christina highlighted the importance of being ‘paradigm literate’ in order to navigate the complexity of public health decision making.

Academic perspective

Our final speaker, Dr Olivia Maynard, gave an academic perspective on how to advise decision makers.

Focusing on practical tips, she outlined her own work on tobacco, smoking, e-cigarettes, alcohol and other drugs and how she has engaged with various opportunities to work with policymakers.

Starting with a clear case for doing the work (it’s important, it’s interesting, to create impact), she went on to outline methods of engagement:

  • Proactively presenting your work; introduce yourself to policymakers interested in your area such as MPs, Peers, APPGs, subject specialists in parliamentary research services, advocacy groups, and PolicyBristol; review Hansard and Early Day Motions; get involved in parliamentary events
  • Respond to calls for evidence (University of Bristol researchers can find curated opportunities via the PolicyBristol PolicyScan)
  • Work directly with policymakers, for example via Policy Fellowships (for example with POST)

Olivia outlined some reflections around the differences between academia and policymaking.

Timelines for action is one, but she also used the changes towards plain packaging as an example to note that the policymaking process can span numerous years, presenting many opportunities for intervention.

She referred back to Christina’s point about ‘multiple competing realities’ to highlight that evidence is one of many factors to consider in policymaking.

She also encouraged academics to challenge ‘imposter syndrome’, by emphasising ‘you are more of an expert than you think you are’, and needing to make yourself known to be offered opportunities.

Where next?

Chair David Buck highlighted a number of themes running throughout the presentations including recognising the paradigms used by different stakeholders; questioning what counts as evidence, and being able to provide advice from an uncertain evidence base; and what these themes mean for all of us (and how willing are we to act on these reflections?)

The seminar concluded with a facilitated Q&A session spanning topics such as:

  • Should all research which influences policy be coproduced with user groups and policymakers?
  • What kind of ‘payback’ do stakeholder organisations need for their involvement in research projects?
  • How should researchers develop the skills needed to cross boundaries?
  • What funding is available for policy relevant research?
  • How can we make our evidence ‘stand out’?
  • Should academics have a responsibility to critique policy?

The seminar started numerous conversations which we hope to continue.

Chair David Buck facilitates our Q&A. Image credit: Julio Hermosilla Elgueta

Access the slides:

1. Richard Gleave Crossing boundaries – undertaking knowledge informed public health

2. Isabel Oliver Partnerships between science and society

3. Christina Gray Evidence into practice

4. Olivia Maynard An academic’s perspective

View a recording of the event on the IEU’s YouTube channel: https://www.youtube.com/watch?v=-ew-RvzV-D0 

Contact lindsey.pike@bristol.ac.uk if you’d like to hear about future events.