Can ChatGPT Write for JournalFeed?

June 26, 2023

Written by Clay Smith

Let’s Talk About AI

I think we need to have an open conversation about the power and usefulness of artificial intelligence (AI), like ChatGPT.  It is not a fad, and this technology is only going to improve over time.  Candidly, ChatGPT is pretty amazing. Thus, I want to be clear on the value we, as human authors, bring to you, the JournalFeed readers.  Spoiler alert – it’s a lot. We are going to perform and format our summaries differently in order to serve you better. I think it’s good to be frank about ChatGPT and how we differentiate what we do vs. what a super-smart computer can do. Here are my thoughts.

ChatGPT as JournalFeed Author?

Here is a recent prompt I put into ChatGPT.  It was the Lancet abstract for the RECOVERY high or low dose dexamethasone article from last week (not picking on any JF author here…just happened to be the article I was editing when I fell down this rabbit hole).

My Prompt:

Would you please summarize the following abstract into a 250 word brief report meant for emergency physicians?

Background: Low-dose corticosteroids have been shown to reduce mortality for patients with COVID-19 requiring oxygen or ventilatory support (non-invasive mechanical ventilation, invasive mechanical ventilation, or extracorporeal membrane oxygenation). We evaluated the use of a higher dose of corticosteroids in this patient group.

Methods: This randomised, controlled, open-label platform trial (Randomised Evaluation of COVID-19 Therapy [RECOVERY]) is assessing multiple possible treatments in patients hospitalised for COVID-19. Eligible and consenting adult patients with clinical evidence of hypoxia (ie, receiving oxygen or with oxygen saturation <92% on room air) were randomly allocated (1:1) to either usual care with higher dose corticosteroids (dexamethasone 20 mg once daily for 5 days followed by 10 mg dexamethasone once daily for 5 days or until discharge if sooner) or usual standard of care alone (which included dexamethasone 6 mg once daily for 10 days or until discharge if sooner). The primary outcome was 28-day mortality among all randomised participants. On May 11, 2022, the independent data monitoring committee recommended stopping recruitment of patients receiving no oxygen or simple oxygen only due to safety concerns. We report the results for these participants only. Recruitment of patients receiving ventilatory support is ongoing. The RECOVERY trial is registered with ISRCTN (50189673) and ClinicalTrials.gov (NCT04381936).

Findings: Between May 25, 2021, and May 13, 2022, 1272 patients with COVID-19 and hypoxia receiving no oxygen (eight [1%]) or simple oxygen only (1264 [99%]) were randomly allocated to receive usual care plus higher dose corticosteroids (659 patients) versus usual care alone (613 patients, of whom 87% received low-dose corticosteroids during the follow-up period). Of those randomly assigned, 745 (59%) were in Asia, 512 (40%) in the UK, and 15 (1%) in Africa. 248 (19%) had diabetes and 769 (60%) were male. Overall, 123 (19%) of 659 patients allocated to higher dose corticosteroids versus 75 (12%) of 613 patients allocated to usual care died within 28 days (rate ratio 1·59 [95% CI 1·20-2·10]; p=0·0012). There was also an excess of pneumonia reported to be due to non-COVID infection (64 cases [10%] vs 37 cases [6%]; absolute difference 3·7% [95% CI 0·7-6·6]) and an increase in hyperglycaemia requiring increased insulin dose (142 [22%] vs 87 [14%]; absolute difference 7·4% [95% CI 3·2-11·5]).

Interpretation: In patients hospitalised for COVID-19 with clinical hypoxia who required either no oxygen or simple oxygen only, higher dose corticosteroids significantly increased the risk of death compared with usual care, which included low-dose corticosteroids. The RECOVERY trial continues to assess the effects of higher dose corticosteroids in patients hospitalised with COVID-19 who require non-invasive ventilation, invasive mechanical ventilation, or extracorporeal membrane oxygenation.

From Higher dose corticosteroids in patients admitted to hospital with COVID-19 who are hypoxic but not requiring ventilatory support (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2023 May 6;401(10387):1499-1507. doi: 10.1016/S0140-6736(23)00510-X. Epub 2023 Apr 13.

Here is what ChatGPT wrote:

Summary for Emergency Physicians:

A recent study called the Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial evaluated the use of higher doses of corticosteroids in patients hospitalized for COVID-19 who required oxygen support. The trial included adult patients with clinical evidence of hypoxia (low oxygen levels) and compared two groups: one receiving usual care with higher dose corticosteroids and the other receiving usual care alone, which included low-dose corticosteroids.

Between May 2021 and May 2022, 1,272 patients with COVID-19 and hypoxia were enrolled in the study. Of these, 1% received no oxygen support, while 99% received simple oxygen only. The primary outcome measured was 28-day mortality. The results showed that in patients who required no oxygen or simple oxygen, higher dose corticosteroids increased the risk of death compared to usual care with low-dose corticosteroids. Specifically, 19% of patients receiving higher doses of corticosteroids died within 28 days, compared to 12% of patients receiving usual care alone.

Additionally, the study found an increase in cases of pneumonia due to non-COVID infections and a higher incidence of hyperglycemia requiring increased insulin doses in patients receiving higher dose corticosteroids.

These findings suggest that for patients hospitalized with COVID-19 who only require no oxygen or simple oxygen, higher dose corticosteroids should be avoided as they increase the risk of death compared to low-dose corticosteroids. However, the study is still ongoing to evaluate the effects of higher dose corticosteroids in patients who require non-invasive ventilation, invasive mechanical ventilation, or extracorporeal membrane oxygenation.

As emergency physicians, it is important to be aware of these findings and consider the appropriate use of corticosteroids in patients with COVID-19 based on their oxygen requirements. Further updates from the RECOVERY trial will provide more insights into the use of higher dose corticosteroids in patients with severe respiratory support needs.

From ChatGPT

It is 299 words, and it is not too badly written.  I would have phrased some things differently and would have included the actual doses of dexamethasone, etc.  But overall, it’s not too bad. And it took about 2-3 seconds.  Here is my point.  These days, a computer can summarize article abstracts pretty well.  Is it as good as a human?  It depends.  Again, this tech is only going to improve over time.  So, what do we, as human doctors and JournalFeed authors, bring to this?  We have to bring to you what ChatGPT cannot.  And what is that?

The Human Touch

Here are some of the things only we, as human doctors, can bring to you, our readers:

• Critical thinking and analysis of the full text
• Spotting bias and flaws or methodological issues
• Applicability to our specific patient population
• Evaluation of clinical importance
• Impact on our practice (or not)
• How we will implement it in practice (if applicable)
• What the findings mean to us personally
• How the article makes us think, feel, or act differently
• What it means for the patients we see each day – how will it potentially impact their lives?
• Humor, inside jokes, “dad jokes,” a clever turn of phrase…instead of blah regurgitation of facts
• We want to write well, with scientific integrity, but we don’t want to be too mechanical.  JF is not a research paper.  It’s a balance between being slightly cheeky yet professional and scientific.  This is, we hope, uniquely human.
• We try to write something that makes you smile (or laugh out loud) or something that touches you or connects with you emotionally in some way.  This requires a human touch and is tough to pull off!

In short, we are rethinking the way we summarize the literature.  If what we are bringing you can be done by a computer, we are not serving you as well as we should be.  So, this is a great opportunity to up our game and serve you better.  

We still want to give you a brief one sentence summary of the meat of the article. But in addition to this, we want to give you a brief insight as to how each article will change our practice – or not. We are dropping the “Why does this matter?” section, and will add this to the body of the post as the first sentence.

We will also work hard to think critically about each article and point out weaknesses, flaws, biases, or methodological issues we see. Of course, I try to not pick crummy articles to start with, but sometimes they get past me. And sometimes I choose articles that I know have issues but still have something important for you or have a good teaching point. To be clear, there is a balance in being critical. If all we do is bash, bash, bash, that gets old in a hurry. We like to emphasize the positive, but we know it lends credibility and trust when we identify and highlight issues with an article, explain them to you, and guide you as to whether or not the research should still change your practice.

What Will This Look Like?

Here is an example I reworked from this past Monday’s post. This new format will roll out over the next few weeks. A few summaries will still have the old formatting since they were written before this change.

---

Spoon Feed
This RCT found high-dose dexamethasone (20mg daily) compared to 6mg/day significantly increased the risk of 28-day mortality in patients hospitalized with COVID-19 who were either mildly hypoxic and on no oxygen or who were on simple supplemental oxygen.

Too much of a good thing?
Early trials in COVID-19 patients found that 6mg of dexamethasone daily reduced mortality among those who were hypoxic.  The RECOVERY group was a randomized controlled trial of 1,272 patients with COVID-19 who were hypoxic and received either no oxygen (1%) or simple oxygen (99%). Patients requiring non-invasive ventilation or mechanical ventilation were excluded. Patients were randomized to either usual care (6mg of dexamethasone daily for 10 days or until discharge) or 20mg PO/IV dexamethasone daily for 5 days followed by 10mg daily for 5 days or until discharge.

The high-dose group had higher 28-day mortality compared to usual care: 19% vs. 12%; RR 1.59 (95%CI 1.2-2.1, p=0.0012) and had a higher rate of progression to any ventilatory support (18% vs. 14%, RR 1.3) and of non-COVID-19 pneumonia (10% vs. 6%). The trial was stopped early due to the signal of harm in the higher dose steroid group.

The study occurred from May 2021 to May 2022, during which the Delta variant was predominant, with Omicron surging at the tail end.  Also, just over half the patients enrolled were vaccinated during this trial, which impacts the generalizability of this study.

How will this change my practice?
I don’t often prescribe inpatient medications for COVID-19 patients, but EPs are increasingly called upon to start treatment in the ED, especially as boarding times increase.  Some EPs even write admission orders after hours.  I will avoid using high-dose dexamethasone in these patients, and stick with the usual, lower dose. It looks like less is more in this case.

Source
Higher dose corticosteroids in patients admitted to hospital with COVID-19 who are hypoxic but not requiring ventilatory support (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2023 May 6;401(10387):1499-1507. doi: 10.1016/S0140-6736(23)00510-X. Epub 2023 Apr 13.

---

Do you see the elements that ChatGPT couldn’t get?  Nowhere in the article was the dominant variant of COVID during the study mentioned (which was an interesting oversight, I thought).  I dug to find a NYT graphic that showed the dominant variant at the time of the study was Delta.  ChatGPT wouldn’t know that this could impact the generalizability of this study, but we would. Also, what about vaccination status and altered boosters that cover the new variants?  Only we, as human doctors who care for patients, would know to look for this.  Also, we would know that EDs are overcrowded, and sometimes we have to start treatment for the admitting team or even write admission orders.  So, only we can speak to how this study personally impacts us and the patients we see.

Conclusion

The technology to summarize the literature already exists and will only improve over time. We, at JournalFeed, want to position ourselves to not only bring you a brief summary of the most important articles for our specialty but to critically analyze them, articulate what they mean to us personally, and walk through how they apply to our practice as we care for patients at the bedside. That is something that ChatGPT can’t do…yet! Our goal is to serve you better as you serve your patients.

AI is a hot topic. What are your thoughts? Please feel free to leave a comment.