The PAPERs Podcast

Episode article

Lingard, L. (2023). Writing with ChatGPT: An illustration of its capacity, limitations & implications for academic writers. Perspectives on Medical Education, 12(1), Article 1.

Enjoy listening to us at your preferred podcast player.

Apple    Google    Spotify    Spreaker 

*** This summary was created by ChatGPT3.5*** 
The article discusses the utility of ChatGPT for academic writing. The author recommends using ChatGPT to support laborious writing tasks and refining its output with one’s unique voice and style for creative writing. 

Episode notes

Background 

I had to do it. Generative AI is forefront in much of my academic thinking these days. I was part of a university task force on AI and Education.  (Opportunity lost when the chair didn’t invite a single computer scientist to join the committee… <face palm>.) So, I’ve been scouring the HPE lit for a paper that allows us to jump into the conversation. Back in late winter we covered one of the first-past-the-post (is that a Canadian idiom?) manuscripts examining Chat GPT (3.0) successful passing of the USMLE.  And the tech has only gotten better since then.  Unfortunately, the literature hasn’t improved much.  Essentially, it’s a series of copycat manuscripts that show the newer tech is better and better at conventional education tasks – mainly assessment.  I haven’t seen much about the use of genAI to improve teaching.  (Disclosure:  I’m working on such a project, so shotgun, I’ve called it everyone else hands off and go find another gen AI project.) 

Enter this commentary by Lingard.  I never choose a commentary, as it doesn’t match with the vibe of the podcast – a dissection of methodology and empiric evidence to inform positions/opinions in HPE.  Unlike a research study, a commentary is all opinion.  But today I’m making an exception, as this commentary provides a worked example of the potential benefits of generative AI and show cases where I think academic writing is going in the next 18 months.  Lingard provides as a counterpoint to the dominant discourse around AI, where Skynet is here and the literal sky is falling (or is it burning?) 

Purpose 

”Drawing on chats I had with ChatGPT4 in March and April 2023 to illustrate its capacity and its limitations, I extract a series of Cautions and Insights and guide writers in how to use incremental prompting to ‘train’ the software, how to use it for brainstorming and generating content like outlines and summaries, and how to employ it as an editor.” 

Lingard (2023)

Methods 

Not applicable… but I’m share Jason or Linda is going to argue with me that this is a case study.  (But the author doesn’t use case study methodology.) 

Results/Findings 

1. You need to train ChatGPT via incremental prompts.

• LLM don’t possess knowledge or search for information.  Based on correlations via reading the “entire” web pre-2021, ChatGPT predicts the next word in a sequence.  It has no concern for accurate, verifiable nor generalizable responses.  It is essentially a text generator. 
• For academic writing this requires subject-matter expertise from the author, to ensure accuracy and completeness (i.e., what wasn’t included) in a response. 
• Training needs to occur with the start of each NEW chat.  (It doesn’t remember what you asked previously. ChatGPTs memory is contextual to the current conversation.) 

2. ChatGPT can help you brainstorm.

• Creating an outline for a paper, which is often the first step in overcoming writer’s block. 
• Create a title for a paper 
• But brainstorming references for writing will (always) lead to invented (but realistic-appearing) sources. 

3. ChatGPT can review your writing and generate counterarguments to strengthen your article 

4. ChatGPT can review your writing and generate a summary (abstract). 

• Including lay summaries that are often required in grant applications, by asking the genAI to adopt a simpler language and explanatory tone. 

5. ChatGPT can provide copy editing to improve writing clarity. 

• It can suggest topic and transition sentences that an author will need to rework into their own voice. 
• For authors, where English is an additional language, ChatGPT can serve as an inexpensive resource to improve readability. 

6. Check your target journal for policies involving generative AI before using ChatGPT. 

Conclusions 

 ”Distinguish the laborious from the creative writing tasks: use ChatGPT to support the former, and keep the latter for yourself. And always view what it has generated as a first draft which you will refine and rework, infusing it with your own particular emphases, your unique voice and style.” 

Lingard (2023).

Paperclips  

This is part of a Perspectives in Medical Education (PME) series called: The writer’s craft.  If you are early in your scholarly career and looking for the best ways to articulate your scholarship- in written form – this series offers ways to structure, articulate, and present your research and your opinion in an engaging and persuasive manner.

Want more? Different aspects of AI in Medical education and Academia can be found in
Papers AI Theme Collection.