Skip to main content

Students Showcase Their Work in Augmented and Virtual Reality at the Qualcomm Institute

Students taught by QI's Neil Smith gathered in Atkinson Hall to present work applying computer science and extended reality to diverse topics.

Published Date

Article Content

On an early evening in June, undergraduate students from academic departments across UC San Diego filed into the Qualcomm Institute’s Atkinson Hall to present final projects applying computer science and extended reality to diverse topics including Maya ruins, California wildfires, and an interactive avatar of Mahatma Gandhi. 

The event was a public showcase from “Digitally Documenting Climate Change” (SYN100) and an offering (AIP197) from the Academic Internship Program, in which students earn academic credit while exploring a professional setting. Both courses were taught by Neil Smith, QI associate research scientist, lecturer with the UC San Diego Departments of Anthropology and Computer Science and Engineering, and co-director of the QI Center for Cyber-Archaeology and Sustainability (CCAS). The Academic Internship Program course was co-taught by QI Director Ramesh Rao.

Smith encouraged students to engage with topics like climate change and the preservation of endangered heritage sites in ways that stoked creativity, using approaches he draws on in his own fieldwork in the Eastern Mediterranean and Levant. 

“The overarching goal is for these two classes to teach students how to employ digital tools such as Unreal Engine to create new media for game-based learning,” said Smith, emphasizing the wide applications for these tools in research and gaming. 

Gaming for Climate and the Environment

Over three quarters, students in “Digitally Documenting Climate Change” developed video games designed to teach a public audience about climate change and socioeconomic impacts. 

A female student plays a virtual reality game.
A student plays a SYN100 virtual reality game on climate change and coral bleaching. Team member Dylan Greher (international business, minor in climate change studies) stands in the background.

Dylan Greher, a third-year undergraduate majoring in international business with a minor in climate change studies, worked with classmates Miranda Angulo, Kida Huang, Yoel Kee and Wilson Ye. The team explored climate change’s effects on diverse communities through a video game set off of the Great Barrier Reef near Badu Island, Australia.

Players don a virtual reality headset to become a citizen scientist tasked with building a machine to lower water temperatures and prevent coral bleaching. Players can swim underwater to collect parts, including a propeller and solar panel, and watch real interview footage of people affected by climate change. On shore, visitors can speak with characters representing a marine biologist, a fisherman and a member of the indigenous community.

“We wanted to share the feel and impact of climate change, instead of focusing on statistics,” said Greher. “Everyone has a different stake in climate change.” 

The Seventh College course also tasked students with taking on new responsibilities such as storyboarding, scripting, coding and even drafting microgrant proposals to fund more complex graphics for the entire class and future classes to come.

Other projects featured quest-type games challenging players to navigate through a maze of wildfires in the California wilderness; collect items on an expedition to underground Maya ruins lying in the path of a large-scale infrastructure project; find their way through a tree maze peppered with dust storms and other hazards of a changing climate in California’s San Joaquin Valley; and learn about environmental and health issues facing communities around the evaporating Salton Sea.

In at least two cases, students built their virtual backdrops using intricate 3D recreations or “digital twins” created by researchers with QI’s Cultural Heritage Engineering Initiative and ALERTCalifornia, a public wildfire safety service also based at UC San Diego. 

“People [at QI] were so into collaborating that we were able to create some really in-depth projects,” said Smith. “All these different groups worked with my students to help them accomplish what they did in the time they had.”

An Immersive Virtual Space

Students in the Academic Internship Program worked with the Cyber-Archeology Warehouse (CyberArchWarehouse), an immersive metaverse space for students and scientists to collaborate on archaeological research from around the world. 

During the showcase, seven students with the CyberArchWarehouse ushered visitors into a room with three large, high-definition screens. On the centermost screen, a digital avatar of Gandhi awaited visitors’ questions, spoken into a microphone on a headset. 

Students gather around a video screen showing an avatar of Gandhi
From left to right: Students Kevin Chan, Joseph Teh, Joseph Dang, Raj Sunku and team lead Sophia Boss created an interactive avatar of Gandhi based on only 36 minutes of footage of Gandhi speaking.

Gandhi’s responses were the work of a suite of artificial intelligence platforms, including Deepface, ChatGPT, Metahumans and more. Working as a group of four programmers and three modelers, the students transformed recordings into a believable likeness that could mimic Gandhi’s voice, accent and personality. Their primary challenge was working with only 36 minutes of footage. None of them had prior coding experience. 

Other projects included a “smart” classroom in VR as well as digital twins of Atkinson Hall. These were displayed on QI’s walk-in virtual reality platform, the SunCAVE, as well as the VROOM, a floor-to-ceiling theater display. 

A fourth team recreated Scripps Institution of Oceanography’s Birch Aquarium. This team spent two hours capturing photos of marine life and the building’s structural features. Team members then used structure-from-motion techniques to bring the reproduction to life, adding to a growing list of virtual museums housed in the CyberArchWarehouse.

The Birch Aquarium’s digital twin is one example of a project students can carry forward this summer if they continue with the CyberArchWarehouse. To create more opportunities to practice the skills in the field, locally, Smith has set out to establish a space for students to partner with departments like Scripps Institution of Oceanography to deliver digital twins for education and exploration. 

Within Atkinson Hall, students will also have access to QI’s powerful wireless infrastructure, which allows them to enter digital spaces like the CyberArchWarehouse’s “smart” classroom from any part of the building as a team. 

To learn more about QI’s growing ecosystem of hands-on academic offerings and internships, visit the Education page.

Share This:

Category navigation with Social links