Evolving Foundations
NCPD Article

A Case for Caution: Patient Use of Artificial Intelligence

Lisa Stewart

Wesley G. Patterson

Christopher Farrell

Janice S. Withycombe

artificial intelligence, cancer biomarkers, chatbot, patient education, ChatGPT
CJON 2024, 28(3), 252-256. DOI: 10.1188/24.CJON.252-256

Artificial intelligence use is increasing exponentially, including by patients in medical decision- making. Because of the limitations of chatbots and the possibility of receiving erroneous or incomplete information, patient education is a necessity. Nurses can advocate for patients by emphasizing the importance of conferring with oncology professionals before making decisions based solely on self-investigation using artificial intelligence.

Jump to a section

    Earn free contact hours: Click here to connect to the evaluation. Certified nurses can claim no more than 1 total ILNA point for this program. Up to 1 ILNA point may be applied to Professional Practice. See www.oncc.org for complete details on certification.

    In the past decade, there has been exponential growth in tumor (somatic) genomic testing. In 2022, the American Society of Clinical Oncology issued a provisional clinical opinion recommending that all patients with advanced or metastatic cancer undergo genomic sequencing if any alterations have regulatory approval to guide the use or exclusion of certain treatments (Chakravarty et al., 2022). The growth in testing also extends to germline testing for cancers such as pancreatic cancer, for which the National Comprehensive Cancer Network (NCCN) recommends germline and genomic tumor testing regardless of the patient’s family history (Crowley et al., 2023). Tissue acquisition has changed too, with the option in many cancers of acquiring a liquid biopsy from a simple blood sample to analyze circulating tumor DNA. The ease of specimen acquisition with liquid biopsy has expanded opportunities to use these diagnostics beyond the initial diagnosis for disease surveillance (including minimal residual disease assessment) and to monitor for treatment effectiveness and resistance.

    Although many alterations are not actionable, the number of targetable alterations continues to increase. Many clinical trials specify alterations in their inclusion criteria, and there has been a shift in trial designs to gene- or biomarker-directed trials (Fountzilas et al., 2022). The expanded use of genomic tumor testing has been accompanied by increasingly complex interpretations (Chakravarty et al., 2022). Alterations may or may not have clinical significance, or significance may be unknown. Molecular testing reports can be challenging for patients, and sometimes clinicians, to understand (Davies et al., 2020).

    At the end of 2022, OpenAI released ChatGPT, version GPT-3.5, to the public for free. Reuters reported on February 1, 2023, that there had been 100 million users of ChatGPT in January 2023, making it the fastest-growing application in history (Hu, 2023). The public has embraced ChatGPT to complete tasks such as writing emails, crafting résumés, and explaining complex topics. Patients are also beginning to report use of artificial intelligence (AI) systems to assist them with understanding medical reports (SOPHIA, 2023). The following fictitious case study showcases an example.

    Case Study

    Becky is a 40-year-old, never smoker, White cisgender female who developed a painful “catch” in her right ribs. A single mother with two daughters aged 10 and 12 years, Becky has a well-paying job in sales that requires some travel, and she describes it as demanding and high pressure. Her primary care provider evaluated her and referred her to pulmonology. Imaging revealed a 4 cm tumor in the right lower lobe, with a metastatic rib lesion. The pulmonologist performed a tumor biopsy, with pathology revealing lung adenocarcinoma with a programmed cell death–ligand 1 expression of 20% and low tumor mutational burden. At the same time, the team ordered a DNA-based next-generation sequencing (NGS) panel on the tumor, finding no identifiable driver alterations. Becky looked at the test results in the hospital patient portal as soon as they became available and proceeded to search the internet and read about non-small cell lung cancer (NSCLC). She felt that all the available sources were complicated and confusing.

    Becky opted to use ChatGPT, version GPT-3.5, to get a more comprehensive description. She asked about the treatment of NSCLC. ChatGPT responded that, generally, treatment consisted of chemotherapy, immunotherapy, and targeted therapy, with the disclaimer that it was not providing medical advice and recommended consultation with a qualified oncologist (OpenAI, 2023). ChatGPT also mentioned clinical trials, interprofessional care, and a periodic reevaluation of the patient’s case and NGS results. Becky then asked ChatGPT for the average survival time for a patient with metastatic lung cancer and no driver alterations (OpenAI, 2023) (see Figure 1 for transcript). ChatGPT responded that the prognosis can vary widely and is challenging, with no targeted treatment options; however, advances in treatment are ongoing, and an oncologist should be consulted for the latest treatment. ChatGPT stated that at its last knowledge update in 2021, the median survival for patients without driver alterations in NSCLC was around 8–12 months with standard chemotherapy. Becky was in shock after reading the prognosis and focused on the time frame of 8–12 months, disregarding the other information and disclaimers.

    FIGURE1

    During the weekend, Becky had time to think about her next steps. She decided to quit her job to spend as much time as possible with her daughters and scheduled a meeting a few days later to submit her resignation. She made appointments with her attorney and financial planner to discuss custody of her children and estate planning. The next day, Becky had her initial visit with her oncologist, Dr. L, who explained that he was suspicious that the tumor might have an oncogenic driver that was not apparent on the initial DNA-based NGS results, particularly with her nonsmoking history (Devarakonda et al., 2021). Dr. L further explained that some driver alterations in NSCLC involve a fusion, which an RNA-based NGS panel may detect (Benayed et al., 2019). Regarding prognosis and treatment options, Dr. L said the conversation would be deferred until all testing was complete because new findings could significantly change treatment options and prognosis. Dr. L ordered the RNA-based NGS panel and discouraged Becky from making any life-altering decisions. Becky heeded the advice and canceled the meeting with her manager.

    At the next visit, Becky received news that the RNA-based panel revealed an NTRK3 fusion partnered with ETV6. Dr. L explained that NTRK fusion alterations in NSCLC are rare but highly responsive to targeted therapy (Liu et al., 2019). Entrectinib, a tyrosine kinase inhibitor that targets tropomyosin receptor tyrosine kinases, showed an overall response rate of 70% at the latest clinical trial update (Demetri et al., 2022). With no central nervous system metastasis evident on Becky’s magnetic resonance imaging, Dr. L explained that survival could possibly exceed three years. They proactively discussed additional treatment options beyond entrectinib, if needed. Becky was excited to hear a better prognosis and realized that she had almost made a huge mistake in prematurely quitting her job. She was thankful that the medical team helped clarify her diagnosis, treatment options, and prognosis.

    Debrief of Case Study

    Becky retrieved her results from the hospital patient portal before her consultation with Dr. L. The release of results to patients in real time remains controversial (Friedman, 2022). As many patients do, Becky proceeded to explore the internet to research her situation without consulting any healthcare professionals. She also turned to a chatbot to answer her questions. Although ChatGPT included statements that cautioned against using the information as medical advice, this may not deter patients from making decisions prior to medical consultation. Before October 1, 2023, ChatGPT output was based on a knowledge cutoff from 2021 (OpenAI, 2023). Oncologic diagnostics and treatments can change significantly in two years, particularly in the genetic and genomic realms. Because ChatGPT did not incorporate current practice recommendation guidelines and did not have the full context of the patient’s clinical picture, the output of the chat conversation resulted in an inaccurate life expectancy.

    The NCCN (2023) Clinical Practice Guidelines in Oncology recommend consideration of RNA-based NGS to maximize the detection of fusion events that initial testing may miss. Eighty percent or more of never smokers (people with a lifetime history of fewer than 100 cigarettes) have clinically actionable driver alterations (Devarakonda et al., 2021). In this case study, ChatGPT told the patient that additional diagnostic testing might be necessary to confirm treatment options.

    Discussion

    AI already has a role in cancer diagnostics by assisting with the technical interpretation of digital images and pathology slides (Rodriguez, 2023). AI could also provide information in different literacy levels and languages for patients who need those accommodations. Ultimately, AI chatbots may take on more mundane administrative tasks such as making appointments, obtaining preauthorization from insurance companies, and serving as administrative assistants (Sava, 2023). Chatbots are an attractive option for patients to privately ask any question without fear of appearing unintelligent or experiencing embarrassment. With unlimited access, patients are able to ask questions at any time of day. Patients can revisit the chat to review the previous conversation if they forget what was said or have additional questions.

    Current publicly available AI interfaces have limitations in medical situations, which can lead to serious consequences. In the case study, ChatGPT had limited ability to contextualize the clinical situation and did not account for the patient’s never smoker status. Bibault et al. (2019) compared a chatbot named Vik to an interprofessional group of oncologists (medical, radiation, or surgical) in explaining 12 common questions asked about breast cancer therapy management. Findings showed Vik to be noninferior to (on par with) the oncologists’ explanations (Bibault et al., 2019). A more recent study comparing the 2021 NCCN guidance with ChatGPT found that ChatGPT suggested at least one NCCN-recommended treatment (Chen et al., 2023). However, 34.3% of ChatGPT outputs also recommended treatments that differed from NCCN guidelines (Chen et al., 2023). A study by Pan et al. (2023) comparing four different chatbots found that chatbots produced accurate information, but that the information was written at a college level and lacked visual aids. These findings showcase the current limitations of AI in that specific prompting (e.g., specification as to reading level) is needed to provide a tailored outcome.

    AI models are only as good as the data they are based on, and they use varying levels and types of human supervision. Neural networks power AI chatbots like ChatGPT to provide responses based on patterns and information from their training (OpenAI, 2023). In the case study, ChatGPT used outdated information, which led to an inaccurate prognosis for the patient. Even with disclaimers, patients may take erroneous or incomplete information as truth. These findings highlight the need for clinicians to be proactive in guiding patients to the right educational resources.

    According to Bibault et al. (2019), the risks of using a chatbot include misdiagnosis, poor treatment adherence, delayed diagnosis, and inappropriate self-medication. Bibault et al. (2019) also recommended that chatbots need to be evaluated like a drug or medical device. Pan et al. (2023) noted that AI chatbots usually mentioned limitations in giving advice and informed users to seek medical advice and attention. They recommended avoiding AI chatbots as a primary source of medical information.

    Implications for Nursing

    Nurses and other healthcare professionals have a crucial role in guiding patients through their diagnoses and treatment journeys. This takes time and may be difficult in a busy work environment. Providing the patient with additional educational resources may help to supplement their learning. AI systems could be used by the healthcare team to enhance patient understanding of oncology-related information. For example, they can use AI in a controlled manner to assist with developing educational materials for patients with lower literacy (Khaja, 2023). Clinicians can point patients toward validated sources of health information (see Figure 2) and provide expert input into developing new educational resources to meet patient needs. Additional measures for nurses to consider include the following:

    • Stay current on clinical recommendations for genetic and genomic testing and its implications for diagnosis and treatment. Nurses are trusted healthcare providers; patients may approach them first with questions related to complex test results.
    • Educate patients at the first oncology visit that information they receive via self-investigation using AI may be inaccurate. Advise patients to avoid making decisions based on this information before conferring with medical professionals. Provide patients with adequate time to ask multiple questions during the initial consultation and at follow-up visits.
    • Consider posting a cautionary warning on the electronic health record patient portal advising patients to seek medical guidance in interpreting all medical results.
    • Recognize that reliable resources are available for clinicians to provide to patients. Use of these materials to assist with patient education is recommended. However, additional information specific to genetic and genomic testing is needed, and nurses can contribute to the development of these future educational materials.

    FIGURE2

    Conclusion

    Patients are using AI more frequently to assist with understanding their diagnoses, including complicated information such as genetic and genomic testing results. It is important that oncology nurses and other clinicians guide patients to validated information and diligently work to develop patient-friendly educational material for complex topics such as understanding genetic and genomic implications for cancer treatment. Additional research is needed on the best methods to assist patients with understanding complex genomic findings.

    About the Authors

    Lisa Stewart, ACNP, AOCNP®, was, at the time of this writing, a doctoral candidate in healthcare genetics/genomics in the College of Behavioral, Social, and Health Sciences at Clemson University in South Carolina and is a field-based senior medical science liaison for Bristol Myers Squibb; Wesley G. Patterson, PhD, MSPA, PA-C, CAQ-Peds, is a genetics physician assistant at Greenwood Genetic Center in South Carolina; and Christopher L. Farrell, PhD, is an assistant professor and Janice S. Withycombe, PhD, RN, MN, FAAN, is an associate professor, both in the School of Nursing at Clemson University. The authors take full responsibility for this content and did not receive honoraria or disclose any relevant financial relationships. Stewart can be reached at lstewa6@clemson.edu, with copy to CJONEditor@ons.org.

    References

    Benayed, R., Offin, M., Mullaney, K., Sukhadia, P., Rios, K., Desmeules, P., . . . Ladanyi, M. (2019). High yield of RNA sequencing for targetable kinase fusions in lung adenocarcinomas with no mitogenic driver alteration detected by DNA sequencing and low tumor mutation burden. Clinical Cancer Research, 25(15), 4712–4722. https://doi.org/10.1158/1078-0432.CCR-19-0225

    Bibault, J.-E., Chaix, B., Guillemassé, A., Cousin, S., Escande, A., Perrin, M., . . . Brouard, B. (2019). A chatbot versus physicians to provide information for patients with breast cancer: Blind, randomized controlled noninferiority trial. Journal of Medical Internet Research, 21(11), e15787. https://doi.org/10.2196/15787

    Chakravarty, D., Johnson, A., Sklar, J., Lindeman, N.I., Moore, K., Ganesan, S., . . . Meric-Bernstam, F. (2022). Somatic genomic testing in patients with metastatic or advanced cancer: ASCO provisional clinical opinion. Journal of Clinical Oncology, 40(11), 1231–1258.

    Chen, S., Kann, B.H., Foote, M.B., Aerts, H.J.W.L., Savova, G.K., Mak, R.H., & Bitterman, D.S. (2023). Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncology, 9(10), 1459–1462. https://doi.org/10.1001/jamaoncol.2023.2954

    Crowley, F., Gandhi, S., Rudshteyn, M., Sehmbhi, M., & Cohen, D.J. (2023). Adherence to NCCN genetic testing guidelines in pancreatic cancer and impact on treatment. Oncologist, 28(6), 486–493. https://doi.org/10.1093/oncolo/oyad044

    Davies, G., Butow, P., Napier, C.E., Bartley, N., Juraskova, I., Meiser, B., . . . Best, M.C. (2020). Advanced cancer patient knowledge of and attitudes towards tumor molecular profiling. Translational Oncology, 13(9), 100799. https://doi.org/10.1016/j.tranon.2020.100799

    Demetri, G.D., De Braud, F., Drilon, A., Siena, S., Patel, M.R., Cho, B.C., . . . Rolfo, C. (2022). Updated integrated analysis of the efficacy and safety of entrectinib in patients with NTRK fusion-positive solid tumors. Clinical Cancer Research, 28(7), 1302–1312. https://doi.org/10.1158/1078-0432.CCR-21-3597

    Devarakonda, S., Li, Y., Rodrigues, F.M., Sankararaman, S., Kadara, H., Goparaju, C., . . . Govindan, R. (2021). Genomic profiling of lung adenocarcinoma in never-smokers. Journal of Clinical Oncology, 39(33), 3747–3758. https://doi.org/10.1200/JCO.21.01691

    Fountzilas, E., Tsimberidou, A.M., Vo, H.H., & Kurzrock, R. (2022). Clinical trial design in the era of precision medicine. Genome Medicine, 14(1), 101. https://doi.org/10.1186/s13073-022-01102-1

    Friedman, D. (2022, October 3). Your medical test results are available. But do you want to view them? The New York Times. https://www.nytimes.com/2022/10/03/well/live/medical-test-results-cures…

    Hu, K. (2023, February 2). ChatGPT sets record for fastest-growing user base—Analyst note. Reuters. https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-…

    Khaja, H. (2023, October 13). Using AI language models to simplify patient education materials. Rheumatology Advisor. https://www.rheumatologyadvisor.com/home/topics/practice-management/use…

    Liu, J., Mu, Z., Liu, L., Li, K., Jiang, R., Chen, P., . . . Wu, D. (2019). Frequency, clinical features and differential response to therapy of concurrent ALK/EGFR alterations in Chinese lung cancer patients. Drug Design, Development and Therapy, 13, 1809–1817.

    National Comprehensive Cancer Network. (2023). NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®): Non-small cell lung cancer [v.3.2023]. https://www.nccn.org

    OpenAI. (2023). ChatGPT (Mar 15 version) [Large language model]. https://chat.openai.com/chat

    Pan, A., Musheyev, D., Bockelman, D., Loeb, S., & Kabarriti, A.E. (2023). Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncology, 9(10), 1437–1440. https://doi.org/10.1001/jamaoncol.2023.2947

    Rodriguez, T. (2023, November 10). AI in oncology: Current and future applications. Cancer Therapy Advisor. https://www.cancertherapyadvisor.com/features/ai-in-oncology

    Sava, J. (2023, April 27). Current applications of artificial intelligence in oncology. Targeted Oncology. https://www.targetedonc.com/view/current-applications-of-artificial-int…

    SOPHIA. (2023, May 21). How I analysed my blood test results with ChatGPT: My personal experience. Medium. https://medium.com/@sophia.sofroniou/how-i-analysed-my-blood-test-resul…