UCSD Students Touch the Future in Collaboration with Museum of Photographic Arts

Article Content

Visitors to a recent technology and art exhibit in Japan were able to use high-tech, touch-screen interface technology to play the role of museum curator, thanks to technology developed in part by two students from the University of California, San Diego.

As part of their research with the Pacific Rim Experiences for Undergraduates (PRIME) program at the UC San Diego division of the California Institute for Telecommunications and Information Technology (Calit2), undergraduates Lance Castillo and Wesley Hsu designed the touch-screen interface for a multi-touch table that allows museum or art exhibit visitors to manipulate digital copies of photographs. The touch-sensitive table (or ‘touch table’) was featured at this summer’s Knowledge Capital Trial Event in Osaka, Japan, which brought together cutting-edge technology and art.

The collaborative project was the first of its kind between UCSD, the San Diego-based Museum of Photographic Arts (MOPA) and the National Institute of Information and Communications Technology (NICT) in Tokyo, Japan, where Castillo and Hsu worked under the guidance of Professor Shinji Shimojo of Osaka University’s Cybermedia Center.

“The goal of the project is to blend the museum field with technology, using UCSD undergraduates to bridge the two,” says Jason Haga, assistant project scientist in UC San Diego’s department of bioengineering and PRIME mentor for the project.

Touch-screen interface

Touch table interface created by Hsu and Castillo allows exhibit patrons to scroll through a subset of MOPA's photographs (left section), which can be dragged to the workspace (middle section) and examined, along with the photograph's metadata. Patrons can drag up to 10 photographs to their "My Gallery" (right section).

The touch-sensitive screen interface at the Knowledge Capital event allowed visitors to view and manipulate digitized images of 50 photographs from MOPA’s collection, ranging from portraits to cityscapes. Users employed hand gestures on the surface of the 2.5’ x 3’ screen (which resembles an overgrown iPad) to choose photographs from a scrollable side bar and drag them to a central workspace for closer inspection.

Within the workspace, photographs can be enlarged and shrunk with expanding and pinching finger movements, respectively. Touching an icon on the side of each photograph provides the name of the photographer and other technical and historical details about the photograph. These features operate with a fluidity and responsiveness that creates a highly immersive experience.

“More than other art forms, photography is a digital medium. It is therefore natural to display our collection using the latest digital presentation technology,” says Amber Lucero-Criswell, Director of Education and Public Programs at MOPA, which is one of only three museums devoted to photography in the US and the only one on the west coast. 

After playing around with the photographs, visitors drag up to 10 favorites into the “My Gallery” portion of the computer interface where the order of the photographs can be finalized and stored along with the visitor’s name and a title chosen for their personalized ‘collection.’
 
“It’s a fun exercise in letting people get a glimpse of how you make a collection, how you group images that have a connection to each other,” says MOPA Deputy Director Vivian Kung Haga about using the exhibit to teach people about the museum curation process.

However, not all visitors are inclined to interact directly with the technology. “We know from numerous studies in the museum field that there are different types of visitors. Some like to participate in a hands-on way and some like to just observe,” says Kung Haga.

Tile wall display of 24 monitors at the Knowledge Capital Trial Event in Osaka, Japan scrolls through patron's "My Gallery" collections. The patron's names (top) and collection title are displayed with each collection.

To engage the visitors who choose to merely observe, Castillo and Hsu connected the touch table to a nearby tile-wall display of 24 computer monitors, which cycled through the saved “My Gallery” collections. The tile-wall display was engineered by the lab manager at NICT, Masaki Chikama, who provided much support to Castillo and Hsu.

The project also marks the first digital exhibition of MOPA’s photographs in an Asian country. “Because of our location, one of our goals was to interact more globally with Pacific Rim countries and also to collect Pacific Rim images,” says Kung Haga.

Building the interface and touch table with open-source resources

Hsu, a senior in cognitive science, drove the design of the touch table interface and Castillo, a senior in computer science, did the majority of the programming for the project. The two worked closely, with Hsu contributing significant portions of the programming code and Castillo collaborating with Hsu on many design decisions.

The students used the Adobe Flash-based Open Exhibits software to develop the interface. Open Exhibits is an open-source software program made by software developer Ideum specifically for interactive museum displays. It is made available for free to approved educational or research institutions thanks to funding from the National Science Foundation.

Wesley Hsu, Jason Haga, Lance Castillo, Joaquin Ortiz, Vivian Kung Haga, Amber Lucero-Criswell (L-R)

One of the challenges of the project was making everything bilingual. From help menus to the photograph descriptions, almost everything was presented in either English or one of three Japanese scripts (hiragana, katakana and kanji), including the touch-screen keyboard that visitors used to type in their names and collection titles. 

The touch table itself was also a product of free, open-source information. Working at NICT last summer under supervision of Shimojo and Calit2 Professor of Visualization and Virtual Reality Falko Kuester, former PRIME student Kevin Nguyen built the touch table according to design blueprints made publicly available by a group of researchers at Cornell University. Nguyen's touch table is also based, in part, on some of the earlier technologies developed by Calit2 researchers Roger Jennings and Kevin Ponto.

The table utilizes FTIR (Frustrated Total Internal Reflection) technology to sense touch. Infrared beams run through a sheet of transparent acrylic. When the surface of the acrylic is touched, the infrared beams are locally deflected downwards, indicating the precise location and timing of the touch. This touch-sensing technology is different, and far cheaper, than the ‘capacitive touch’ technology of the iPad or iPod that relies on sensing the small electrical currents in users’ fingertips.

Nguyen’s acrylic touch screen was mounted into a wooden frame positioned above a standard video projector to display all of the interface graphics, bringing construction of the touch table to under $1,000. For comparison, store-bought touch tables with similar capabilities can run several thousand dollars. 

Haga says that the low cost of the hardware, coupled with the free Open Exhibits software “makes this technology available to most museums that would otherwise not be able to afford expensive off-the-shelf touch tables.”

Future plans

MOPA plans to bring in Hsu and Castillo as consultants to install and update their touch-screen interface on a larger, sleeker version of the touch table for a Fall 2012 exhibit at MOPA. The touch screen will display more photographs from MOPA’s collection, as well as audio and video clips related to the photographs. MOPA officials plan to crowd source the curation process for the exhibit by having visitors, as well as users of its website, select their favorite photos.

Meantime, the project team will process surveys and a mountain of user data collected and stored by the touch table in Osaka to discover patterns in how users picked and ordered the photographs and to find out which were the most and least popular. This information will be used to improve the visitor experience for MOPA’s Fall 2012 exhibition.

Along with Haga and Joaquin Ortiz, Digital Interpretation Manager at MOPA, Hsu is already planning a return to Japan this month to demonstrate the touch-screen interface at the biannual PRAGMA (Pacific Rim Applications & Grid Middleware Assembly) workshop in Sapporo. The project team will also be submitting reports of their work to a handful of upcoming museum conferences.

This project is made possible by a grant from the U.S. Institute of Museum and Library Services and support from NICT. PRIME is made possible by funding from the U.S. National Science Foundation, with additional funding from NICT and MOPA. Additional support was provided by the Legler Benbough Foundation, and the Downing Family Foundation.

About PRIME: The PRIME program provides undergraduates the opportunity to do real research while living for nine weeks in more than half a dozen Pacific Rim countries (and India), working with mentors at both the host institution and at UCSD, where the faculty participants come from a variety of departments and research units on campus including Calit2.

The goal of PRIME, now in its eighth year, is to better prepare students to work in the global economy and the research environments of the future. More than 150 students hosted by 14 institutions along the Pacific Rim have participated in the PRIME program, which is funded by the National Science Foundation (NSF), with additional support from Calit2.