Tools for assessing for equity

In my country one can no longer do health research without considering matters of equity. This is a good thing. Because of inequities in the health system much of the data - including in research data sets - will reflect those inequities. Risk models, clinical decision pathways and the like derived from that data will likely reflect and even reinforce those inequities - usually to the detriment of minority groups. Knowing this I want to develop a suite of analytical tools to assess for inequity and to try to minimise reflecting inequities in any models I develop. I would very much appreciate suggestions of approaches/techniques at the various stages of the research process, namely:

  1. Epidemiological tools for assessing inequities in existing health data
  2. Tools for use when sampling that may account for underlying inequity
  3. Modelling methods which may account for inequities
  4. Analysis techniques which assess model performance in sub-groups
  5. At the implementation stages any special techniques to ensure fairness

I don’t want to pre-empt this with my own suggestions - simply initiate a discussion. It would help if any replies state which of the five stages a tool may apply.

Many thanks.

5 Likes

I’m really glad to see this question, and I am looking forward to the discussion. I have been trying to learn about all of this as well, and one thing I am learning is that it’s important to step back and learn about the historical/social context in which the inequities developed before trying to apply any tools; otherwise, we can wind up making things worse.

Here are a few resources I have found helpful, in no particular order:

  • The We All Count project for equity in data science, https://weallcount.com. Not necessarily geared towards academic research, but they provide a good framework and other resources.
  • Data Feminism by Catherine D’Ignazio and Lauren F. Klein, https://mitpress.mit.edu/books/data-feminism. I am currently in Chapter 2 and highly recommend the book so far. The book uses the word “feminism” in a very broad sense: “… throughout the book, we employ the term feminism as a shorthand for the diverse and wide-ranging projects that name and challenge sexism and other forces of oppression, as well as those which seek to create more just, equitable, and livable futures.” (p. 6)
  • Fatal Invention: How Science, Politics, and Big Business Re-create Race in the Twenty-first Century by Dorothy Roberts, https://thenewpress.com/books/fatal-invention. An excellent overview of what exactly race is and how scientists have (intentionally or not) perpetuated racism in the name of science.

I will add other resources later in the week, but I wanted to go ahead and suggest these while I was thinking about it.

3 Likes

Excellent, thank you Lauren.

A useful resource is The AI Fairness 360 toolkit GitHub - Trusted-AI/AIF360: A comprehensive set of fairness metrics for datasets and machine learn which is a collection of tools developed by the research community to quantify and mitigate bias in data or machine learning models.

4 Likes

This looks very interesting— thank you. One thing I’m learning from Data Feminism (link is above) is that the concept of equity goes beyond fairness and the removal of bias. I am still only in Chapter 2, but I am learning a ton and definitely recommend the book so far!

1 Like

It looks like this online workshop might address some of your questions: Advancing Health Equity Methodologies and Approaches, https://iaphs.org/tools-for-success/preconference-workshop-2/. I won’t be able to attend but would love to hear about it if anyone else goes.

Well spotted Lauren - I too would be interested in hearing from anyone attending.

1 Like