By Ella Kipling
While artificial intelligence continues to rise in prominence and people turn to the technology for medical help, women and people of colour have been warned about following advice from chatbots. Magali Geens, who led the STADA Health Report 2025 on health trends in the UK and Europe, explained that AI will not always provide accurate health advice for women and anyone who isn鈥檛 white. When we ask AI systems like Chat GPT for guidance , it will pull together studies and data from the internet to provide its response. However, this existing knowledge is often outdated. A large amount of the research into health issues has historically been centred around men, with them often being the default subjects in studies. Caucasians are also overrepresented in the available studies on the internet that Chat GPT uses to form its advice. Geens explained: 鈥淚f [women] use Chat GPT and put in symptoms, you will get data and information that is not for you.鈥 She said that to get the best out of AI when it comes to health advice, you will need tools that are up-to-date, relevant, and that eliminate bias. Geens used specific chatbots designed for mental health problems as a good example. Using AI like a search engine to ask for medical advice will result in you getting inaccurate information, or results that will confirm bias. You would need to provide more details, such as your height, weight, family history, and gender to get a tailored response – but many AI users are unaware of this. While AI can be a useful tool for those in rural areas to access health advice, it is vital for users to remain critical of the information they are presented with. Geens told us: 鈥淭his is a challenge that will take a few more years to crack.鈥 Peter Goldschmidt, CEO of pharmaceutical company STADA, added that AI can provide 鈥渋nformation, but not competent judgement鈥. Women already have less trust in healthcare systems than men. According to the STADA survey of 27,000 people from 22 counties, 55% of women trust healthcare systems, compared to 61% of men. AI continues to rise in popularity and play a role in education, science, and now, healthcare. Of those surveyed, 39% said they would consider getting a medical consultation from AI instead of seeing a doctor – with most citing accessibility and convenience as the reasons why. The technology is already in use in the UK healthcare system, with the UK Health Security Agency harnessing AI to understand patient experiences, detect food-borne illness outbreaks, and make public health guidance more consistent. Dr Nick Watkins, chief data scientist at UKHSA, said that AI could 鈥渆nhance public health protection鈥. He added: 鈥淎s we continue to develop and refine these systems, we maintain a careful balance between embracing innovation and ensuring robust validation of AI outputs. This approach helps us harness AI’s potential while maintaining the high standards expected of a national public health agency.鈥