When ChatGPT first entered the mainstream and the team at the Penn Center for Learning Analytics gathered to think about its uses in education, the immediate dream was simple: Eliminating student questions about the syllabus.
What they’ve managed to accomplish, however, exceeded expectations.
“We spend a surprising amount of time [on the syllabus] in class,” says Ryan Baker, a professor of learning, teaching, and literacies, and director of the Penn Center for Learning Analytics (PCLA) in the Graduate School of Education. “Could we take basic class information and put it into the knowledge bases and use it? That was the original idea, and we tried it in a course and found it was useful for way more than that kind of information—and in fact, that wasn’t where it was best.”
The result of the experiment is JeepyTA, a virtual teaching assistant developed at the Center and used by instructors, teaching assistants, and students. JeepyTA incorporates manually uploaded course materials into the large language model GPT from OpenAI. While JeepyTA can field questions about a syllabus, Baker says they’ve found the tool to be most helpful for formative feedback—feedback that “helps students think about things,” he explains.
Baker says JeepyTA has also been impressive with developing personas. In one case, students in a course at the University of Wisconsin related to culturally responsive pedagogy used the tool to interview and ask questions. JeepyTA aggregated interviews from real people to build the personas.
Its use cases have expanded as faculty at GSE and other universities have expressed interest in incorporating the tool. Baker has used it in several of his own courses, and other faculty members have used it in ones ranging from project-based to programming-based.
“I’ve lost track of the number of faculty using it and the number of courses—it’s quite a few,” he says.
Work on JeepyTA began in summer 2023 as a collaboration between Baker and Senior Researcher Maciej Pankiewicz and has been iterative as faculty have provided feedback. Hundreds of students have used the tool and completed surveys about their experience. The intent, so far, is to use it as a research tool to learn how artificial intelligence can improve the course experience for everyone and reduce tedious time-consuming tasks for instructors and teaching assistants. Think: That weeklong pile-up of emails to send to students on a Sunday morning.
Keenly interested in helping with development of an AI teaching assistant tool is Rachel Liu, a first-year doctoral candidate in GSE who began working on the project as a master’s student in learning analytics. A math and economics major as an undergraduate, she came to Penn to leverage her skills with coding and statistics for education. Broadly, she’s interested in how to better support students and instructors using the massive amount of data created in the online environment.
“We are using data to derive some insights to inform better teaching strategies, better scaffolds for students, and [that work is] grounded by learning theories,” Liu says of those researching learning analytics. “So, it is quite interdisciplinary. This field is also rapidly advancing with the AI coming out.”
Today, JeepyTA looks, she says, “nothing like the original version.” The team has made changes to its interface and functionality after receiving feedback. They also survey students at the beginning of a course to understand average expectations of an AI virtual assistant, collect survey results once the semester ends, and have occasionally adjusted the tool mid-semester to accommodate needs that arise in the moment. Instructors test the tool for quality at the beginning of the semester, receiving sample responses and adjusting the tool as needed.
Liu says it’s also been a priority to offer alternatives to JeepyTA, as well as answer questions about privacy. (All students can opt out from using JeepyTA.) The intent of the tool is not to replace human instruction, she emphasizes. For certain types of queries, students only receive a response once a TA or instructor has reviewed and had an opportunity to amend the automatically generated response.
“It is our responsibility to talk to the student [if they have concerns] and, if they request an alternative, we should make that happen and provide support in a way that feels comfortable for them,” she says. “We hope JeepyTA can support student learning, but it’s not the only resource they can use. We want to be mindful of that.”
She and members of the PCLA are also working to involve students in co-designing JeepyTA’s features so that the tool reflects their learning needs, addresses their concerns, and supports their academic work in ways that feel useful and relevant.
Next, adds Liu, the team of approximately seven people working on the project hope to develop a richer understanding of how students are using the tool through interviews inspired by Liu’s education in Penn GSE instructor Amanda Barany’s Quantitative Ethnography course.
“We probably need to continue doing survey interviews for a couple rounds before we can truly understand what is the best way to design this platform,” Liu says. “And also, ultimately, we hope the JeepyTA platform is user-friendly, so people with minimal background in AI will be able to use it.
“We want to make JeepyTA as accessible as possible for students from different backgrounds, while still maintaining the quality of the AI-generated outputs.”
Artificial intelligence touches disciplines across campus. In a limited spring profile series, Penn Today is highlighting innovative students at Penn who are adopting this technology in a variety of projects. To learn more about how members of the Penn community are pioneering the understanding and advancement of AI, visit the Penn AI website.
