Abstract

Artificial intelligence in our day and age is becoming more and more common as better AI models come out every month. It has the potential to improve clinical decision-making, diagnosis, and patient outcomes; however, an AI system that performs well overall may not perform equally well for every patient population. This project looks at whether it is ethical to use AI in healthcare to improve outcomes for most patients when it may perform worse for certain populations. Current research shows that differences in training data, health disparities, model design, and data can contribute to unequal AI performance, and studies have found examples of AI systems underdiagnosing underserved populations and performing differently across populations based on race, gender, and skin tone, producing biased recommendations that can influence a healthcare provider's decision. These differences raise ethical concerns related to justice, beneficence, and nonmaleficence because an AI system may benefit many patients while placing certain populations at a greater risk of harm. According to current research, AI systems can still be ethically used in healthcare, but healthcare organizations have a responsibility to identify and reduce disparities before and after implementation. In order to responsibly implement AI, there needs to be testing across different patient populations using more representative data while also being transparent about the limitations of the technology. Overall, healthcare organizations should not rely solely on an AI system's overall accuracy when determining whether it is safe or ethical to use. Instead, they should also consider how the technology affects individual patient populations to ensure that advances in AI do not worsen existing healthcare disparities.

Keywords: artificial intelligencehealthcare disparitiesalgorithmic biashealth equitynursing ethics

References

  1. Abràmoff, M. D., Tarver, M. E., Loyo-Berrios, N., Trujillo, S., Char, D., Obermeyer, Z., Eydelman, M. B., Foundational Principles of Ophthalmic Imaging and Algorithmic Interpretation Working Group of the Collaborative Community for Ophthalmic Imaging Foundation, Washington, D.C., & Maisel, W. H. (2023). Considerations for addressing bias in artificial intelligence for health equity. npj Digital Medicine, 6, Article 170. https://doi.org/10.1038/s41746-023-00913-9
  2. American Nurses Association. (2010). Nursing’s social policy statement: The essence of the profession (3rd ed.).
  3. American Nurses Association. (2025). Code of ethics for nurses. https://codeofethics.ana.org/home
  4. California Board of Registered Nursing. (n.d.). Nursing Practice Act. California Department of Consumer Affairs. Retrieved November 2, 2026, from https://www.rn.ca.gov/practice/npa.shtml
  5. Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., Mukherjee, P., Phung, M., Yekrang, K., Fong, B., Sahasrabudhe, R., Allerup, J. A. C., Okata-Karigane, U., Zou, J., & Chiou, A. S. (2022). Disparities in dermatology AI performance on a diverse, curated clinical image set. Science Advances, 8(32), eabq6147. https://doi.org/10.1126/sciadv.abq6147
  6. Elgendi, M., Elkhalifa, A., Khandoker, A., Maruotti, A., Zhou, H., Menon, C., & Ward, R. (2026). Dataset effects outweigh algorithmic effects in determining fairness of healthcare machine learning. npj Digital Medicine, 9, Article 571. https://doi.org/10.1038/s41746-026-02723-1
  7. Haider, S. A., Borna, S., Gomez-Cabello, C. A., Pressman, S. M., Haider, C. R., & Forte, A. J. (2026). The algorithmic divide: A systematic review on AI-driven racial disparities in healthcare. Journal of Racial and Ethnic Health Disparities, 13(1), 188–217. https://doi.org/10.1007/s40615-024-02237-0
  8. The Holy Bible, English Standard Version. (2016). Crossway. (Original work published 2001)
  9. Jabbour, S., Fouhey, D., Shepard, S., Valley, T. S., Kazerooni, E. A., Banovic, N., Wiens, J., & Sjoding, M. W. (2023). Measuring the impact of AI in the diagnosis of hospitalized patients: A randomized clinical vignette survey study. JAMA, 330(23), 2275–2284. https://doi.org/10.1001/jama.2023.22295
  10. Lett, E., Shahbandegan, S., Barak-Corren, Y., Fine, A. M., & La Cava, W. G. (2025). Intersectional and marginal debiasing in prediction models for emergency admissions. JAMA Network Open, 8(5), e2512947. https://doi.org/10.1001/jamanetworkopen.2025.12947
  11. Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., & Mathur, P. (2023). Bias in artificial intelligence algorithms and recommendations for mitigation. PLOS Digital Health, 2(6), e0000278. https://doi.org/10.1371/journal.pdig.0000278
  12. OpenAI. (2026). ChatGPT [Large language model]. https://chatgpt.com/
  13. Saha, A., Shao, L., Kim, J., & Dunn, A. G. (2026). Fairness in multimodal machine learning applications in clinical decision support: A systematic review. npj Digital Medicine. Advance online publication. https://doi.org/10.1038/s41746-026-03000-x
  14. Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182. https://doi.org/10.1038/s41591-021-01595-0
  15. Sjoding, M. W., Ansari, S., & Valley, T. S. (2023). Origins of racial and ethnic bias in pulmonary technologies. Annual Review of Medicine, 74, 401–412. https://doi.org/10.1146/annurev-med-043021-024004
  16. Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021). Mitigating bias in machine learning for medicine. Communications Medicine, 1, Article 25. https://doi.org/10.1038/s43856-021-00028-w
  17. Yang, Y., Zhang, H., Gichoya, J. W., Katabi, D., & Ghassemi, M. (2024). The limits of fair medical imaging AI in real-world generalization. Nature Medicine, 30(10), 2838–2848. https://doi.org/10.1038/s41591-024-03113-4