Mellon Foundation Supports New Software Tools for Humanities Researchers

Article Content

Lev Manovich and a visiting researcher exploring a visualization of one million manga pages on HIPerSpace visualization system constructed by Gravity Lab at Calit2.

Computers have changed the landscape of humanities research. Innovations continue to make it cheaper and easier to digitize and analyze ever larger volumes of data. But most e-humanities tools focus on manuscripts and other textual records. Now researchers at the University of California, San Diego are working to enable widespread exploration of big image and video collections, too. With support from a $477,000 grant from the Andrew W. Mellon Foundation, the researchers will develop new software for analyzing large visual data sets and integrate it into SEASR/Meandre, a technology platform designed for humanities research.

Software Studies researchers working with visualizations of hundreds of video.

“The digital humanities have been developing very rapidly in the last few years, with many new centers, projects and jobs,” said Lev Manovich, professor of visual arts at UC San Diego and principal investigator on the Mellon Foundation grant. “The ability to analyze and find patterns in large data has already transformed many areas of sciences and business. This grant from the Mellon Foundation will allow us to construct comprehensive and easy-to-use software tools that will allow humanities researchers to explore visual data sets that until now seemed too big to handle.”

In the past, the Mellon Foundation has provided key support for the development of SEASR/Meandre, with the desire to eventually see the platform evolve to include processing of images and video. For its part, the UC San Diego-based Software Studies Initiative, led by Manovich, has been at the forefront of building tools for exploring and visualizing large visual data sets for humanities applications. Those free tools have been used by scholars in disciplines ranging from art history and archaeology to film and media studies, dance studies and game studies.

According to Manovich, researchers currently do not have adequate software for exploring large collections of images. Standard web interfaces show only a handful of images at a time. The image display is limited to fixed formats – typically grids, lists, or slide shows. “They don’t allow you to see a collection as a whole, or interactively sort it by different criteria to make visible the underlying trends in the visual data,” explained Manovich. With the software to be developed with the Mellon Foundation grant, he said, “with a single click, a researcher will be able to start a whole series of steps which will run automatically – downloading images from a collection, processing them to extract hundreds of visual features from each image, doing statistical analyses, and generating a series of visualizations.”

Visualization by UCSD undergraduate student Megan O'Rourke comparing representation of faces in 316 paintings by Impressionist artists.

In addition to Manovich, the UC San Diego team includes Jeremy Douglass, who leads software development in the Software Studies Initiative, and Elizabeth Losh, director of academic programs in UC San Diego’s Sixth College. They will collaborate with other humanities researchers, including Loretta Auvil, co-PI on the SEASR project based at the University of Illinois at Urbana-Champaign, and Almila Akdag, a postdoctoral researcher in the e-Humanities Group of the Royal Netherlands Academy of Arts and Sciences, on large-scale case studies.

Three case studies are planned. Each will demonstrate how, within a particular field, quantitative analysis and visualization of images or video can open new research possibilities for that field. One case study will continue analysis of one million pages of manga (Japanese comics); another will analyze one million images from deviantArt (the most popular social network for non-professional art); and the third will apply the new tools to thousands of hours of TV programs and online video.

Over the past 10 to 15 years, libraries, museums and other cultural institutions have digitized massive volumes of images and video. For example, the Historical American Newspaper archive in the Library of Congress contains high-resolution digital scans of 4,800,000 pages of U.S. newspapers, while the Museum of Modern Art in New York and other museums make available online images of all the art works in their collections.

Manovich’s group has successfully used its software with several different types of image collections and has been approached by a number of prominent cultural institutions and collections, including the Library of Congress, the Getty Research Institute, the Austrian Film Museum, the Netherlands Institute for Sound and Image, and Magnum Photos.

“They are interested in using our techniques with their collections and data sets,” Manovich said. “Our tools will allow them to make use of massive amounts of digitized visual information to make new discoveries and to create more inclusive histories.”

Added Manovich: “With this new grant, we will be able to fully integrate our techniques and tools with SEASR/Meandre, giving its users the ability to do quantitative visualization and analysis of millions of images and videos.”

The Software Studies Initiative – with initial support from the UC San Diego division of the California Institute for Telecommunications and Information Technology (Calit2) and the Center for Research in Computing and the Arts (CRCA) – has been developing a comprehensive set of software tools for the quantitative analysis and visualization of large collections of images and video since 2007.

Visualization by UCSD undergraduate student Megan O'Rourke comparing selected paintings by Impressionist artists.

Manovich noted that the Software Studies Initiative paved the way for the Mellon Foundation grant thanks to prior support for software development and large-scale visualization from the National Endowment for the Humanities (NEH); National Science Foundation (NSF); University of California Humanities Research Institute (UCHRI); and Calit2. He also credits previous campus support in the form of a UC San Diego Chancellor’s Collaboratories grant, which provided funding for graduate students from interdisciplinary backgrounds to work on new visualization software.

The new grant will allow the Software Studies Initiative to expand its outreach to undergraduates at UC San Diego. More than 30 Interdisciplinary Computing and the Arts (ICAM) majors have worked in the lab to date, and the Mellon Foundation grant will allow the team to hire a few ICAM undergraduates to work alongside faculty, postdoctoral researchers and graduate students on the project.