Students Showcase Their Work in Augmented and Virtual Reality at the Qualcomm Institute

Dylan Greher, a third-year undergraduate majoring in international business with a minor in climate change studies, worked with classmates Miranda Angulo, Kida Huang, Yoel Kee and Wilson Ye. The team explored climate change’s effects on diverse communities through a video game set off of the Great Barrier Reef near Badu Island, Australia.

Players don a virtual reality headset to become a citizen scientist tasked with building a machine to lower water temperatures and prevent coral bleaching. Players can swim underwater to collect parts, including a propeller and solar panel, and watch real interview footage of people affected by climate change. On shore, visitors can speak with characters representing a marine biologist, a fisherman and a member of the indigenous community.

“We wanted to share the feel and impact of climate change, instead of focusing on statistics,” said Greher. “Everyone has a different stake in climate change.” 

The Seventh College course also tasked students with taking on new responsibilities such as storyboarding, scripting, coding and even drafting microgrant proposals to fund more complex graphics for the entire class and future classes to come.

Other projects featured quest-type games challenging players to navigate through a maze of wildfires in the California wilderness; collect items on an expedition to underground Maya ruins lying in the path of a large-scale infrastructure project; find their way through a tree maze peppered with dust storms and other hazards of a changing climate in California’s San Joaquin Valley; and learn about environmental and health issues facing communities around the evaporating Salton Sea.

In at least two cases, students built their virtual backdrops using intricate 3D recreations or “digital twins” created by researchers with QI’s Cultural Heritage Engineering Initiative and ALERTCalifornia, a public wildfire safety service also based at UC San Diego. 

“People [at QI] were so into collaborating that we were able to create some really in-depth projects,” said Smith. “All these different groups worked with my students to help them accomplish what they did in the time they had.”

An Immersive Virtual Space

Students in the Academic Internship Program worked with the Cyber-Archeology Warehouse (CyberArchWarehouse), an immersive metaverse space for students and scientists to collaborate on archaeological research from around the world. 

During the showcase, seven students with the CyberArchWarehouse ushered visitors into a room with three large, high-definition screens. On the centermost screen, a digital avatar of Gandhi awaited visitors’ questions, spoken into a microphone on a headset. 

Gandhi’s responses were the work of a suite of artificial intelligence platforms, including Deepface, ChatGPT, Metahumans and more. Working as a group of four programmers and three modelers, the students transformed recordings into a believable likeness that could mimic Gandhi’s voice, accent and personality. Their primary challenge was working with only 36 minutes of footage. None of them had prior coding experience. 

Other projects included a “smart” classroom in VR as well as digital twins of Atkinson Hall. These were displayed on QI’s walk-in virtual reality platform, the SunCAVE, as well as the VROOM, a floor-to-ceiling theater display. 

A fourth team recreated Scripps Institution of Oceanography’s Birch Aquarium. This team spent two hours capturing photos of marine life and the building’s structural features. Team members then used structure-from-motion techniques to bring the reproduction to life, adding to a growing list of virtual museums housed in the CyberArchWarehouse.

The Birch Aquarium’s digital twin is one example of a project students can carry forward this summer if they continue with the CyberArchWarehouse. To create more opportunities to practice the skills in the field, locally, Smith has set out to establish a space for students to partner with departments like Scripps Institution of Oceanography to deliver digital twins for education and exploration. 

Within Atkinson Hall, students will also have access to QI’s powerful wireless infrastructure, which allows them to enter digital spaces like the CyberArchWarehouse’s “smart” classroom from any part of the building as a team. 

To learn more about QI’s growing ecosystem of hands-on academic offerings and internships, visit the Education page.