How Professors Are Embracing ChatGPT and Artificial Intelligence in the Classroom
In the 1968 classic movie 2001: A Space Odyssey, a spacecraft’s artificial intelligence–powered computer system named HAL 9000 begins to act a bit too sentient. When HAL decides humans’ commands are not consistent with its programming, it decides to act independently.
Of course, science fiction is just that—fiction—and nothing quite like the plot of the film has come to pass. But it certainly feels as if we’re a lot closer to hearing a computer say, “I’m sorry, Dave, I’m afraid I can’t do that” than we’ve ever been. That’s because on November 30, 2022, the AI-powered, machine-learning, natural text–producing bot ChatGPT took the world by storm. A year later, AI seems to be everywhere.
You can now use AI to help you write a church talk in the style of Ernest Hemingway. You can use AI to create a photorealistic image of something that never happened. You can use AI to create a new song that sounds just like it was written by Taylor Swift—but wasn’t.
“We’re just seeing the tip of the AI iceberg,” says David Wood, Glenn D. Ardis Professor in the BYU Marriott School of Accountancy. “What’s coming next is going to be exciting or terrifying, depending on your perspective.”
Wood can make such pronouncements because he’s become a bit of a self-proclaimed AI enthusiast during the past year.
In just a few months, he’s published half a dozen papers on what ChatGPT can do in the realm of accounting, including a landmark paper that features the contributions of 327 coauthors from 186 educational institutions in 14 countries. He’s found that ChatGPT can now pass accounting certification exams (which wasn’t the case the first go-round), he’s produced research that demonstrates how ChatGPT can be used in internal auditing, and he’s recently finished a paper on how the bot can handle case-based research (very well, thank you).
Wood is also developing lectures about ChatGPT, with plans to sprinkle them throughout the accounting program. And as a textbook author, he’s including ChatGPT in the next edition. Suffice it to say, Wood’s perspective is very much on the “what’s coming next is exciting” side concerning the future of AI.
“Everyone seems to be taking the angle, ‘How are students going to use AI to cheat?’” Wood says. “Opportunities to cheat have always existed. I think the more interesting angle is ‘How can we use this to make learning better?’”
How It Started
While the modern form of artificial intelligence has existed for only about five years—and ChatGPT has existed about one—its origins date back nearly 75 years. In a paper published in 1950, mathematician and computer scientist Alan Turing wrote about his famous “imitation game” (later called the “Turing Test”) to determine if machines could exhibit intelligent behavior.
The basic idea is that a human and a computer are placed behind a wall, and a second human—the evaluator—asks questions of each through written communication. If the evaluator can’t tell which responses came from the person and which came from the computer, then the machine is considered to have passed the test.
It’s fitting, then, that Wood’s first published foray into artificial intelligence and accounting was a nod to the Turing Test: pitting humans against ChatGPT on accounting exams. And while he didn’t sit behind a wall, Wood and his colleagues did feed accounting assessment questions—a lot of questions—to the chatbot to see how it could manage.
When Wood first decided to pursue the idea a few weeks after ChatGPT launched, he reached out to a few fellow researchers. Colleague and coauthor Jeffrey Pickerd got Wood’s email on December 21, 2022.
“There were only a couple of us involved in the project at that point,” says Pickerd, an associate professor of accountancy at the University of Mississippi. But when Wood posted an open invitation to researchers online, recruitment took off. “A week or two later, that jumped to 150 interested researchers. In the end, we published a paper with 327 coauthors, which is just absolutely unheard of,” Pickerd says.
To put that in perspective, even five coauthors is generally unheard of in accounting papers. Wood believes the response was a clear signal that faculty worldwide want to understand and to contribute to AI’s impact in the classroom.
Crowdsourcing the data set was a unique approach, Pickerd says, but it allowed the collection of an enormous amount of data, which lent significant academic heft to the final product. The 300-plus coauthors provided more than 25,000 classroom accounting exam questions for ChatGPT to answer, while BYU undergraduate students fed the bot an additional 2,268 textbook test bank questions. The questions covered the gamut of subject matter (accounting information systems, auditing, financial accounting, managerial accounting, tax accounting) and varied in difficulty and type (e.g., true/false, multiple choice, short answer).
Impressively (and as a result of Wood’s commitment to work fast), the paper wove its way through the entire research and publication process in less than four months. The motivation, of course, was to publish before the research was obsolete—and with good reason: OpenAI, the maker of ChatGPT, launched its newest version of the AI chatbot product, GPT-4, the very day the article was accepted for publication.
But that didn’t stop the research from getting global attention. In addition to national headlines from Fox Business, Yahoo! News, and the Week, the paper was covered by 50 other outlets across the world, including every major national media outlet in India. The findings were fascinating: while ChatGPT was extraordinary, it still couldn’t outperform students. Students scored an average of 77 percent; ChatGPT managed only 47 percent.
The study revealed some other interesting aspects of the AI bot—at least the earliest version of it—namely, that ChatGPT scored decently on true/false questions (68.7 percent) but struggled with short-answer questions (scoring between 28.7 percent and 39.1 percent). It also invented facts occasionally and provided justifications and explanations that were likewise incorrect, sometimes even producing real-looking but completely fabricated references.
And, in general, it was really bad at math.
“It’s not perfect; you’re not going to be using it for everything,” says Jessica Wood, a BYU undergraduate (and Wood’s daughter) who helped with the study. “Trying to learn solely by using ChatGPT is a fool’s errand.”
How It’s Changing
The highly publicized BYU paper came out April 18, 2023. Now, just months later, it’s an understatement to say that AI has advanced significantly.
“It was funny because on LinkedIn everyone was like, ‘Oh see, we don’t have to worry about anything. ChatGPT is not that smart,’” Wood says. “And then a new edition came out and people thought, ‘Uh-oh.’ It suddenly got a lot smarter.”
And while ChatGPT was getting smarter, AI applications were exploding across the world. According to a Vanity Fair story published September 13, 2023, there were more than 14,700 AI startups in the United States and around 58,000 worldwide. The International Data Corporation, a global tech analytics and consulting firm, reported that revenues from AI are expected to eclipse $500 billion this year.
While it has plenty of AI competition, ChatGPT continues to get its fair share of headlines. A few titles from one recent search-engine query include “Is ChatGPT a Better Entrepreneur than Most?”
“ChatGPT May Diagnose Emergency Room Patients as Well as Doctors,” and “Can ChatGPT Forecast Stock Price Movements?”
In the realm of accounting, Wood and his coauthors always knew it was only a matter of time before ChatGPT evolved. As one commenter on the Fox Business story said, “If the database doesn’t have the accounting rules, then of course it can’t do the work. Feed it the data from the top 100,000 private earners and top 10,000 corporation earners, and once it has the type and source of documents needed, it will outclass any student.”
Since the initial study, Wood has followed up with five more multiauthored working papers, including one showing that the latest version of ChatGPT (GPT-4) successfully passed all sections of the CPA, CMA, CIA, and EA certification exams.
“It has advanced considerably,” says Wood, who remains optimistic about future academia with ChatGPT in hand. “We’re using it in everything: school, teaching, research. Everything we do now has a ChatGPT lens to it.”
How It’s Going
Wood and colleague Scott Summers, the Andersen Foundation Alumni Professor in the School of Accountancy, certainly aren’t shy about welcoming ChatGPT into the classroom. The duo is using it to generate student practice problems. They have also added prompt engineering—an emerging field focused on entering prompt text into ChatGPT and other language-learning bots—to the coursework. Specifically, if a student takes Accounting 407: Data Analytics in Accounting as part of the junior core at BYU Marriott, the second day of class will now include instruction on prompt engineering and large language models.
Additionally, Wood incorporated the technology into coursework early in 2023, with students creating content for TechHub, a BYU-sponsored website that offers programming-language and technology-software challenges.
Moving forward, the professors will be constantly assessing where ChatGPT is a fit—“It’s great to generate, audit, and annotate code,” Wood notes—and where it’s not. In their opinion, it’s not even possible to have a policy at the class level that covers every legitimate use; rather, faculty need to consider if it is appropriate at the assignment level.
Of course, not all faculty are ready to buy in. Wood and Summers took it upon themselves to evangelize at a recent BYU Marriott faculty meeting, teaching a session where they demoed ChatGPT use to colleagues.
“A number of faculty who are close to retirement weren’t as excited by the technology,” Wood says. “But others went back to their offices and signed up for a subscription immediately. For the majority, once they see what it can do, they think, ‘Oh, now I get it.’”
The ChatGPT evangelizing hasn’t been limited to BYU. Wood and Summers occasionally teach seminars to outside faculty and professionals on the accounting merits of the AI chatbot as well. They recently taught 100 faculty from across the country and asked the fellow academics up front if they thought the tech should be banned from academia. The early response was decidedly opposed, with 85 percent saying it should be banned. But after two hours of watching what ChatGPT could do, the attendees had an almost complete reversal of opinion: only 10–15 percent still felt it should be banned.
Pickerd, one of Wood’s longtime colleagues, is among those converted. “There are academics who see ChatGPT as the harbinger of evil and doom, but I think we have to accept it as it is,” Pickerd says. “We can’t just put our blinders on and ignore it. It behooves people entering any industry to see how they can use ChatGPT and other AI tools to increase the effectiveness and efficiency of their work.”
Like Wood and Summers, Pickerd feels strongly that the first step should be teaching students how to use the technology ethically. A simple copy-and-paste approach to writing assignments using ChatGPT does students no good, they argue. But using it to generate early drafts and then taking those drafts and refining and rethinking and analyzing them to produce a final product? That’s different.
For Ann Dzuranin, KPMG Endowed Professor of Accountancy at Northern Illinois University, there are bigger reasons to embrace the technology than just improving student learning—although she agrees with that reason too. To her, the fact that all the Big Four firms have adopted AI is clear support for the need to teach future professionals how to use it.
“It would be discouraging if the firms were saying, ‘No, we’re not going to use this,’ but they’re embracing it,” says Dzuranin, one of the 327 coauthors on the mega paper. “It’s similar to how the Big Four embraced data analytics about 10 years ago. We remember trying to beat the drum back then as a lot of faculty and schools thought it would be a passing fad. But it wasn’t. We do our students a disservice if we don’t help them understand AI and use it wisely.”
When they were new technologies, 10-key machines, calculators, computers, and even Google were accompanied by a wave of fear, but, ultimately, each opened up a new realm of possibilities for more efficient and effective work. “This is just a new tool,” Summers says. “We can’t run away from it; we need to run into it.”
After all, ChatGPT is no HAL. At least not yet.
______
Written by Todd Hollingshead
Editor’s Notes on Illustrations: Marriott Alumni Magazine fed Midjourney, an artificial intelligence image generator, a series of prompts to conjure up images to accompany this article. Diving into the world of AI-generated art was an adventure to say the least. Many of the results were impressive, others were a bit creepy, and some were downright bizarre. These images are illustrations themselves of what it’s like to work with AI. (Though we’re pretty confident these programs and prompts will get better with time.)
About the Author
Todd Hollingshead is a media relations manager in BYU’s University Communications office. He lives in Springville, Utah, with his wife, Natalie; their four children; and a dog and a cat. He swears he didn’t use ChatGPT to write this article.