Mind the gap: The case for sex and gender-aware mental health prediction models

By Francesca Romana Cavallo

---

Statistical Methods at DSxHE: Blog Series on Methods for Health Equity

The Statistical Methods section of Data Science for Health Equity (DSxHE) is launching a public-facing blog series that demystifies methods and showcases their real-world impact on health equity. Each post will blend clear, accessible explanation with concrete applications, from study design and causal inference to measurement, modelling, and evaluation, highlighting how methods can drive fairer health outcomes. We invite authors to bring not only their technical expertise but also their personal insights, and encourage anyone interested to volunteer their perspective. Submissions can be tutorials, case studies, opinion articles, or "what we learned" stories.

Want to contribute? Send a brief pitch or draft to info@datascienceforhealthequity.com.

Editorial note: Posts in this series are written by volunteers. The views expressed are those of the authors and do not necessarily reflect the views of Data Science for Health Equity (DSxHE).

---

What if we could spot the warning signs of a mental health crisis before it escalates? What if we could predict when someone is likely to relapse after treatment? And what if we could identify who might benefit most from early intervention programmes? This is the promise of clinical prediction models: tools that use information like age, medical history, and laboratory test results to estimate someone’s risk of developing a condition or needing care in the future.

In mental health care, these tools might help predict things like the chance of a first episode of psychosis, a relapse of depression, or the risk of hospitalisation. Some models may give a simple score, like a percentage chance of a depressive episode in the next year, while others sort people into categories of risk, such as low, medium, or high, based on their predicted probability of an outcome. In mental health services, prediction models can guide medical decision-making in many ways. Some are designed to help clinicians and mental health professionals make decisions during a consultation, for example whether to refer a patient for specialist psychiatric care or crisis intervention; others act more as decision aids, supporting decision-making by providing personalised mental health risk estimates that may guide conversations between patients and healthcare providers. Ultimately, prediction models in mental healthcare aim to help identify people who might need psychological, psychiatric or social support sooner, making care more timely, targeted, and effective.

These models are already being used in mental health care. For example, OxMIS (Oxford Mental Illness and Suicide tool) helps clinicians estimate the 12-month suicide risk for people with severe mental illness, such as schizophrenia or bipolar disorder (Fazel, 2019). Another tool, P-Risk, is designed for use in GP practices to flag individuals at risk of developing psychosis using routine data from electronic health records, like frequent visits for anxiety or sleep problems (Sullivan 2022).

In recent years, prediction models have become increasingly important in the push for more targeted care, both in primary care settings and mental health contexts. But there’s a catch: for these models to truly serve everyone, they need to be fair. In the context of clinical prediction models, fairness refers to the principle that prediction models should perform well across different population groups, without systematically disadvantageing anyone based on characteristics like sex, gender, ethnicity, or socioeconomic status. In this article, I will focus on sex and gender, where the former refers to the physical and biological characteristics that distinguish males and females, while the latter refers to the socially constructed roles, behaviours, activities, and attributes that a given society at a given time considers appropriate for men and women (UN Women Training Centre, 2021). Sex and gender play a critical role in how mental health conditions appear, are diagnosed, and handled. Risk prediction models that ignore these differences risk missing key patterns or reinforcing existing inequalities.

Mental health care needs to be sex and gender sensitive

We know that mental health does not look the same for everyone, and that sex and gender play a big role in shaping those differences. Research shows that the paths people take toward mental health problems can vary depending on both biological factors, like hormones, and social ones, like gender roles and expectations (Hosang 2018). 

Take depression and anxiety, for example. These conditions are more commonly diagnosed in women, who also tend to experience a greater impact on their daily lives (Whiteford 2013). This difference is partly driven by biological differences like genetics, reproductive health, and hormonal changes (Hyde 2020). But it is also shaped by the wider environment: women are more likely to face gender-based violence, discrimination, and economic inequality, all of which can take a toll on mental health (Oram 2017).

Despite growing awareness, most clinical guidelines for mental disorders still don’t offer meaningful advice on how to tailor care based on sex and gender (Lenz, 2025). Even when gender is mentioned, it’s still too often treated as a checkbox rather than a lens through which to understand someone’s experience. An analysis of neuroscience and psychiatry papers published between 2009 and 2019 showed that only 19% of the studies used a design for discovery of possible sex differences, and only 5% of the studies analysed sex as a discovery variable (Rechlin 2022). Prediction models are no exception. While it is common to include sex or gender as predictors of risk, model development studies rarely use sex or gender-specific models, despite the fact that they tend to perform better for many conditions compared to whole-cohort models (see for example Elliot 2024). If we want truly personalised mental health care, we need models that take into account the different ways mental health conditions manifest in different subgroups, and the unique risk factors that accompany them.

When prediction models overlook gender, they risk painting an incomplete, or even misleading, picture of mental health conditions. That is because gender influences not just who is at risk, but how mental health conditions show up, how people seek help, and how they respond to treatment. If a model is trained mostly on data from men, for example, it might miss the signs of depression in women, who are more likely to report mood and somatic symptoms, while men often present with externalising behaviours such as substance misuse and antisocial traits (Piccinelli 2000). Alternatively, it might underestimate the risk of psychosis in women, whose early warning signs often differ from those seen in men. These kinds of blind spots can have real consequences: it means some people may not be flagged as high risk when they should be, therefore delaying care, misdirecting resources, or reinforcing existing inequalities. 

Understanding algorithmic fairness

Before we can discuss the technical need for gender-aware modelling in mental health, we need to define what a “fair” prediction model actually means. It turns out that there is no single answer, but understanding the different definitions of algorithmic fairness and their trade-offs is key to building more equitable mental health prediction tools. It also turns out that many definitions of algorithmic fairness are focused on achieving equality, instead of equity.

Most fairness metrics aim to achieve equality, i.e., ensuring that different groups are treated the same way by the model. For example, in the context of predicting first psychosis or relapse, demographic parity requires that people from different groups are assigned to risk categories at equal rates (i.e., when flagged as high-risk, the probability of actually having psychosis should be similar across groups). Equalised odds require that the model’s error rates (false positives and false negatives) are similar across groups; for instance, that women are not more likely than men to be falsely reassured that they are at low risk of suicide, or that men are not more likely to be falsely flagged as at high risk of hospitalisation compared to women. And predictive parity requires that the predicted risk scores are equally accurate for different groups, so for example that a given risk score corresponds to a similar probability of developing a mental health condition, regardless of gender.

Demographic Parity (Statistical Parity)

For a predicted outcome $\hat{Y}$ and groups $g_i$, demographic parity is defined as

$$\mathcal{M}_{SP} = \frac{P(\hat{Y} = 1|g_0)}{P(\hat{Y}=1|g_1)} $$

The closer $\mathcal{M}_{SP}$ is to 1, the fairer the classifier.

Example: A model is built to predict who is at risk of psychosis. Demographic parity asserts that the proportion of men and women that are flagged at risk of psychosis is equal.

Considerations: Demographic parity only considers positive predictions, and does not consider if the prevalence of psychosis is different for men and women. If more women are at risk of psychosis, then more women should be flagged as being at risk. Using demographic parity in this case would result in equal treatment rates but disparate outcomes. 

Equalised odds

Equalised odds are achieved when both True Positive Parity, also known as Equal Opportunity, and False Positive Parity are achieved.

For a real outcome $Y$, predicted outcome $\hat{Y}$ and groups $g_i$, True Positive Parity is defined as

$$\mathcal{M}_{TPP}=\frac{P(\hat{Y} = 1|Y=1,g_0)}{P(\hat{Y}=1|Y=1,g_1)} $$

While False Positive Parity is defined as

$$\mathcal{M}_{FPP}=\frac{P(\hat{Y} = 1|Y=0,g_0)}{P(\hat{Y}=1|Y=0,g_1)} $$

The closer both $\mathcal{M}_{TPP}$ and $\mathcal{M}_{FPP}$ are to 1, the fairer the classifier.

Example: A model is built to predict who is at risk of psychosis. We have true positive parity if among people who actually go on to develop psychosis, men and women are flagged as high risk at similar rates; we have false positive parity if among people who do not develop psychosis, men and women are incorrectly flagged as high risk at similar rates. Equalised odds holds if both true positive parity and false positive parity hold.

Considerations: Equalised odds enforce similar error rates across subgroups.

Predictive Parity

Predictive value parity is achieved when both Positive Predictive Value Parity and Negative Predicted Value Parity are achieved.

For a real outcome $Y$, predicted outcome $\hat{Y}$ and groups $g_i$, Positive Predictive Value Parity is defined as

$$\mathcal{M}_{PPV-P}=\frac{P(Y= 1|\hat{Y} =1,g_0)}{P(Y=1|\hat{Y} =1,g_1)}$$

While Negative Predictive Value Parity is defined as

$$\mathcal{M}_{FPP}=\frac{P( Y= 1|Y=\hat{Y},g_0)}{P(Y=1|\hat{Y}=0,g_1)} $$

The closer both $\mathcal{M}_{PPV-P}$ and $\mathcal{M}_{FPP}$ are to 1, the fairer the classifier.

Considerations: Predictive parity is closely related to calibration. If a model is well-calibrated, the proportion of men and women with the same risk score that go onto developing psychosis should be the same, i.e., 

$P(Y=1| \hat{Y}=0.2, g=Men) = P(Y=1| \hat{Y}=0.2, g=Women) = 20\%$, where Y is actual future psychosis and Y is the predicted risk. Equal calibration across groups is important when implementing thresholds, because if $P(Y=1| \hat{Y}=0.2, g=Men) = 30%$ and $P(Y=1| \hat{Y}=0.2, g=Women)=10%$, a threshold of 0.2 will over-diagnose women and under-diagnose men. 

Example: A model is built to predict who is at risk of psychosis. If among people flagged as high risk, the proportion who actually go on to develop psychosis is similar for men and women, we have positive predictive value parity; if among the people not at risk, the proportion who do not go on to develop psychosis is similar for men and women, then we have negative predictive value parity.

Despite the availability of objective metrics, algorithmic fairness in mental health is not a given. A recent study analysing 13 prediction models for psychosis and functional outcomes found that fairness across gender was often not adequately assessed; and when it was, gender biases emerged (Sahin, 2024). For example, some models showed systematically higher positive predictive values for men, meaning they were more likely to be flagged as high risk, even when actual outcomes and risk symptoms did not differ significantly between genders. This kind of imbalance can lead to over-monitoring of some groups and under-supporting others who may struggle to access help, potentially perpetrating existing biases.

These definitions are useful, but they come with trade-offs and, in many cases, it’s mathematically impossible to satisfy all fairness criteria at once. To see how this can happen, imagine a model that predicts suicide attempts in the next year, with a true positive rate (TPR) of 80% and a false positive rate (FPR) of 10% for both men and women. Given that women are more likely than men to attempt suicide (NHS Digital, 2025), let’s imagine that in a group of 1000 women 100 attempt suicide (10%), and in a group of 1000 men 50 attempt suicide (5%). With the given TPR and FPR, the model would have correctly identified 80 women (true positives), and incorrectly labelled 90 as high risk (false positives). For men, there would have been 40 true positives and 95 false positives. From these, we can calculate the positive predictive value for women as 47% and the positive predictive value for men as 30%. So, despite the model showing equalised odds for men and women (in virtue of the same TPR and FPR), the different rates of suicide attemps make it impossible to obtain the same predictive parity between the groups.

More importantly, these metrics focus on equal treatment, not equitable outcomes. They assume that fairness can be achieved by balancing numbers, without considering the wider context: structural inequalities in accessing services, bias in diagnosis and treatment, historical exclusion of women from clinical research, and the different ways mental health conditions manifest across genders. A model might treat men and women “equally” according to a fairness metric, but still fail to capture gender-specific risk factors—like hormonal changes, reproductive health experiences such as perinatal health or menopause, or exposure to gender-based violence—that shape the risk of developing mental health outcomes, how they present, and how likely someone is to receive care.

Algorithmic fairness is just one side of the coin: the goal goes beyond achieving equality to also achieve equity in mental health prediction and care. That means developing tools that account for sex- and gender-specific factors, helping us understand both biological differences and the influence of gender norms on health outcomes, reflecting diverse populations, confronting historical biases, and guiding more equitable health practices.

Methodological approaches

Fairness metrics help us evaluate whether prediction models treat different groups equally, but equality isn’t always enough. Another, complementary way of building fair and trustworthy models is through interpretability, that is the ability to understand and explain how a model makes its predictions or decisions. In risk prediction modelling, explainability techniques help surface important predictors, which, together with fairness metrics, can help assess the model’s validity and fairness (although one should be cautious with attributing causal value to these explanations). For example, they can help explain what factors contribute to disparity measures, or how predictor importance varies across sex and gender. As we’ve seen, mental health conditions don’t manifest the same way across genders, and models that ignore these differences risk missing key signals. For instance, depressive symptoms may present as irritability or substance use in some men, and as anxiety or withdrawal in some women, while trauma‑related symptoms may be more strongly linked to gender‑based violence in women and to community or state violence in some men and gender-diverse people. If fairness metrics are applied on their own rather than as part of a framework reflecting lived experience in the modelling process, they fall short of capturing the complexity of the real world. This is why when we design predictive tools we need to keep in mind that different groups may need different approaches to achieve comparable outcomes. In modelling terms, this means designing tools that do not just treat everyone the same, but instead account for sex- and gender-specific factors that shape risk, symptoms, and care pathways.

When it comes to making prediction models more sensitive to sex and gender, there’s no one-size-fits-all solution. Different approaches make different trade-offs between simplicity, accuracy, fairness, and inclusivity. Below I describe three ways of approaching the model development process, each with its own strengths, limitations, and implications for sex and gender fairness and equity.

Single model with sex or gender as predictors

A single model is trained on data from all individuals, with sex or gender included as one of the input variables. The idea is that the model will learn how sex or gender influences risk, alongside other factors like age or medical history, which can include factors like psychiatric history, medication use, previous hospitalisations, or social determinants. A model predicting relapse of depression after discharge might take as inputs the number of previous episodes, current medication, measures of social support, and gender. The model is then free to assign more or less importance to gender depending on how predictive of depression relapse it is in the cohort. Fairness constraints may be applied during training or post-processing, for example by setting different decision thresholds for each gender or sex, so that the false negative rates for identifying depression are similar across groups, or by calibrating the predicted risks separately in men and women.

While this is the simplest approach to implement and is efficient with limited data, it has a few shortcomings. It assumes that the same set of predictors, with the same functional form, works well across groups. In practice, the relationship between predictions and outcomes may differ: for example, substance use may be a stronger predictor of a mental health crisis in men than women, while reproductive health may be particularly important in women. Also, because the model development is driven by overall model performance, gender-specific patterns can be diluted or missed. If certain gendered experiences, such as domestic violence or perinatal complications are poorly recorded, the model will primarily learn patterns that reflect the majority group. Finally, simply including sex or gender as a predictor, does not address how structural inequalities shape who receives a diagnosis, is admitted to hospital, or gets coded in the record–and the model may inadvertently learn and reinforce those biases.

Gender-specific models

In gender-specific modelling, separate models are trained for different groups; for example, one model for men and another for women. This allows the use of different features and modelling choices for each model, enabling the learning of patterns that are unique to that group, potentially improving accuracy and relevance. A model for women at risk of depression may include variables related to reproductive and menstrual health, while for men it may focus on employment status and substance use. The strengths of these types of approaches are the capturing of unique risk factors for each group, and potentially optimised performance within each group, which may translate into more accurate identification of those at high risk and more relevant prompts for clinicians.

However, there are also important limitations. Separate models require larger training cohorts: splitting the data by sex or gender effectively reduces the amount of information available to each model, which is especially problematic in mental health where outcomes can be rare. Moreover, they can exclude or misclassify gender-diverse individuals if only binary categories are used. Gender-diverse people may be grouped with one of the binary categories, forced into an “other” category with very small numbers leading to unstable models, or excluded altogether, which undermines equity. Also, this approach is more complex to implement and maintain, as clinical systems have to monitor and update several models over time, thereby increasing the risk of errors. Risk scores are not directly comparable across models, especially if using different sets of predictors, which complicates adoption of prediction models for clinical decision making.

Hybrid and alternative approaches

Hybrid approaches sit between single models and stratified models. For example, models can be trained jointly but include interaction terms between gender and other predictors, allowing the model to learn gender-specific effects but still sharing information across groups. Another option is stratified evaluation: training a single model across all individuals, but assessing its performance separately across gender groups to identify and address disparities. For example, a model predicting relapse of psychosis might show good overall accuracy, but might miss a substantial proportion of women who later relapse, or over-flag men as high risk for relapse. Similarly, stratified interpretability analyses can help identify any gender-specific predictors. Techniques such as feature importance or counterfactual explanations can be applied separately within subgroups to see whether different variables drive the same risk score. Transformer-based models trained on electronic health records offer a promising alternative to traditional machine learning models requiring feature engineering. By treating patient histories as sequences of diagnoses, medications, and life events over time, these models preserve the temporal and contextual richness of individual mental health trajectories. They also reduce the need to explicitly specify factors associated with the average risk for the outcome. Instead, they can learn nuanced patterns from the data itself, including how gendered experiences shape mental health trajectories. This opens up exciting possibilities for more personalised and less biased predictions, especially when combined with stratified evaluation and interpretability techniques.

Conclusion

Gender-aware prediction models are necessary for equitable mental health care. When done well, they can identify at-risk individuals earlier, reduce over-medicalisation of some groups, enable more informed shared decision-making, and reveal knowledge gaps that drive better research. Although modelling techniques that are aware of group differences are necessary, they are not sufficient. To truly advance equity, we must also address biased training data, structural barriers to care, and the broader social determinants of mental health. This starts with better data collection. We need to go beyond binary sex markers and collect information of gender identity, reproductive health factors, experiences of discrimination, and data on the wider socioeconomic context. We need to oversample underrepresented groups, for example non-binary folks, whose context is inevitably excluded when reducing their experiences to the limiting information provided by sex assigned at birth.

Next, fairness and explainability must be core design principles. Developers should routinely report performance stratified by sex and gender, and apply multiple fairness metrics tailored to the context in which the model will be used. If models are built for single-sex or gender groups, transparency and justification should be standard practice. Finally, post-deployment evaluation must become the norm. Prediction tools should be monitored in real-world settings to ensure they perform equitably across populations, and adjusted when they don’t.

Ultimately, gender-aware prediction models should be part of a closed feedback loop. Not just between data and deployment, but between modelling, clinical care, and research. These tools can help generate insights to understand how mental health conditions differ across sexes and genders, as well as providing decision-making support. By revealing patterns in symptom presentation, treatment response, and risk factors, gender-aware models can inform future studies, refine clinical guidelines, and shape more inclusive care pathways. In this way, they go from being just predictive, to becoming transformative.

---

Thank you for reading this blog post! What did you find interesting? What questions do you have? Leave your comments down below!

---

Glossary

Disparate impact: Refers to practices that appear neutral but disproportionally affect members of a protected class. For example, if a hospital uses an algorithm to prioritise patients for access to psychosis interventions, but the algorithm is trained on data reflecting the fact that men are more likely to be admitted due to clinician bias or different help-seeking behaviours, then men will be assigned higher risk scores than women. Despite not having any explicit discrimination, women will have worse access to psychosis interventions.

Disparate treatment: Intentional discrimination where individuals are treated differently based on protected characteristics. For example, imagine a care provider using an algorithm to choose whether to refer a patient to a specialist mental health assessment. Two patients, a man and a woman, get assigned similar scores by the algorithm. If the provider decides to refer the man to a depression clinic, but not the woman, attributing her risk score to “normal stress” or “relationship issues”, then we have a situation of disparate treatment.

False negatives: Cases where a test or model incorrectly predicts a negative result for an individual who actually has the condition or outcome of interest. For example, a person with a disease is classified as disease-free.

False positives: Cases where a test or model incorrectly predicts a positive result for an individual who does not have the condition or outcome of interest. For example, a healthy person is classified as having a disease.

Negative Predictive Value: The proportion of individuals with a negative test result who truly do not have the condition. It represents how likely it is that a negative result reflects the true absence of the condition.

Positive Predictive Value: The proportion of individuals with a positive test result who truly have the condition. It represents how likely it is that a positive result reflects the true presence of the condition.

Protected characteristics: Personal characteristics on the basis of which it is against the law to discriminate. Under the UK Equality Act 2010, the protected characteristics are age, disability, gender reassignment, being married or in a civil partnership, being pregnant or on maternity leave, race including color, nationality, ethnic or national origin, religion or belief, sex, sexual orientation.

Stratification: The process of dividing a study population into distinct subgroups (strata) based on specific characteristics such as age, ethnicity, or sex.

Further reading

• The Lancet, A series on Gender Equality, Norms, and Health

• Algorithmic fairness in artificial intelligence for medicine and healthcare | Nature Biomedical Engineering

• Algorithmic Fairness (Stanford Encyclopedia of Philosophy)

References

Fazel S, Wolf A, Larsson H, Mallett S, Fanshawe TR. The prediction of suicide in severe mental illness: development and validation of a clinical prediction rule (OxMIS). Transl Psychiatry. 2019;9(1):98. Published 2019 Feb 25. doi:10.1038/s41398-019-0428-3

Hosang GM, Bhui K. Gender discrimination, victimisation and women’s mental health. The British Journal of Psychiatry. 2018;213(6):682-684. doi:10.1192/bjp.2018.244

Lenz B, Derntl B. Sex-sensitive and gender-sensitive care for patients with mental disorders. Lancet Psychiatry. 2025;12(4):244-246. doi:10.1016/S2215-0366(24)00330-4

Şahin D, Kambeitz-Ilankovic L, Wood S, et al. Algorithmic fairness in precision psychiatry: analysis of prediction models in individuals at clinical high risk for psychosis. The British Journal of Psychiatry. 2024;224(2):55-65. doi:10.1192/bjp.2023.141

Sullivan SA, Kounali D, Morris R, et al. Developing and internally validating a prognostic model (P Risk) to improve the prediction of psychosis in a primary care population using electronic health records: The MAPPED study. Schizophr Res. 2022;246:241-249. doi:10.1016/j.schres.2022.06.031

Gender equality glossary [internet]. New York: UN Women Training Centre; 2021. Available from: https://trainingcentre.unwomen.org/mod/glossary/view.php?id=36.

Rechlin, R.K., Splinter, T.F.L., Hodges, T.E. et al. An analysis of neuroscience and psychiatry papers published from 2009 and 2019 outlines opportunities for increasing discovery of sex differences. Nat Commun 13, 2137 (2022). https://doi.org/10.1038/s41467-022-29903-3

Joshua Elliott, Barbara Bodinier, Matthew Whitaker, Rin Wada, Graham Cooke, Helen Ward, Ioanna Tzoulaki, Paul Elliott, Marc Chadeau-Hyam, Sex inequalities in cardiovascular risk prediction, Cardiovascular Research, Volume 120, Issue 11, July 2024, Pages 1327–1335, https://doi.org/10.1093/cvr/cvae123

Piccinelli M, Wilkinson G. Gender differences in depression: Critical review. British Journal of Psychiatry. 2000;177(6):486-492. doi:10.1192/bjp.177.6.486

Chapter 4: Suicidal thoughts, suicide attempts and non-suicidal self-harm - NHS England Digital