The Happiness of Cities: Do Happy People Take Happy Images?

Article Content

From left, Mehrdad Yazdani, Lev Manovich and Jay Chow.

A team of researchers from the University of California, San Diego and The Graduate Center, City University of New York (CUNY) is one of only six groups to win one of Twitter’s inaugural #DataGrants. To do so, they beat out more than 1,300 rival proposals from around the world.

The pilot program, announced in February, will give the six winning teams free access to Twitter’s public and historical datasets for the purpose of advancing their respective research efforts.

The project will tackle a simple question: Is it possible to measure the overall happiness of metropolitan areas based on the study of images shared on Twitter? “Can visual characteristics of images shared on social media tell us something about the ‘moods’ of cities?” said principal investigator Mehrdad Yazdani, a data scientist with the Software Studies Initiative (SSI) at the Qualcomm Institute, the UC San Diego division of the California Institute for Telecommunications and Information Technology (Calit2). “We will analyze one million tweeted images over the course of one year in specific U.S. cities, and test for correlations with other measures of happiness in the same cities.”

Since only six teams were awarded grants, the Twitter program became one of the most competitive research contests, with the winners representing less than one-half of one percent of the submitted proposals. “We believe that images can transcend the limitations of language,” said Yazdani. Co-PI Lev Manovich, a CUNY Graduate Center professor of computer science who established the Software Studies Initiative at the Qualcomm Institute in 2007, adds: “By studying large sets of images shared on social media, we hope to develop new ways to understand the well-being of a society, and the what, when and where of a society’s needs.”

In addition to Yazdani and Manovich, the winning team includes Nadav Hochman, a Ph.D. candidate in the History of Art and Architecture at the University of Pittsburgh, who is currently working with Manovich as a visiting scholar at CUNY; and Jay Chow, web developer at the San Diego-based startup Katana Connect, who is also a UCSD alumnus (BA, ’12) from the Interdisciplinary Computing and the Arts program, and a volunteer researcher in SSI at the Qualcomm Institute. (SSI originated in the institute, where Manovich remains an academic participant, in addition to running a parallel group at The Graduate Center, CUNY.)

Existing measures of happiness are based on traditional surveys and other data sources, including statistics about crime, health and well-being. To the best of the team’s knowledge, no similar study has been done with images shared on social media, but it’s a logical extension of work done by Manovich’s teams on both coasts. Earlier in 2014, they released a study of 3,200 Instagram ‘selfies’ (self portraits) across six cities In 2013, they analyzed and visualized 2.3 million Instagram photos from 13 cities in their Phototrails project. (For the Phototrails, the team downloaded data and images using the Instagram API; for Selfiecity, they used Gnip, the largest provider of social media data in the world.)

“We will systematically evaluate many visual features of the tweeted images,” explained principal investigator Yazdani, an alumnus (Ph.D., ’12) from the Electrical and Computer Engineering department in UC San Diego’s Jacobs School of Engineering. The researchers will use standard measures used in image processing and computer vision, such as color measures, edge orientations, and texture characteristics,

“We can then test if the characteristics of such images can be correlated with measures of social health such as the Gallup well−being index and Health Ranking,” added Yazdani. “For example, do cities that are more ‘happy’ have more selfies, and do people smile more when taking selfies?”

In addition, the researchers will study image content using machine learning techniques. They will also closely look at selfies shared on Twitter using the methodology developed in the Selfiecity project, including proportions of smiling faces, and the ratio of selfies to all other images.

“We are thrilled with the response from the research community,” said Twitter on its Engineering Blog April 17. “We received more than 1,300 proposals from more than 60 different countries, with more than half of the proposals coming from outside the U.S.” Twitter made only two grants to U.S. institutions; the other four went to teams in Japan, Australia, the Netherlands and United Kingdom.

For Twitter, the #DataGrants program aims to be a public service as well as a showcase for the value of its data. According to the social network, users are sending more than 500 million tweets every day – an average of 5,700 tweets per second (based on data from August 2013).

Twitter’s announcement came just days after the company announced that it has acquired Gnip, a Colorado-based startup that has been an outside partner and go-between for external companies that wanted access to Twitter’s data. (The Software Studies Initiative has already been using Gnip since September 2013.) In announcing its first #DataGrants, Twitter noted that “as we welcome Gnip to Twitter, we look forward to expanding the Twitter #DataGrants program and helping even more institutions and academics access Twitter data in the future.”

The other recipients of Twitter #DataGrants will use the data in research ranging from health and the environment to disaster response and sports. The only other U.S.-based team is a partnership of Harvard Medical School and Boston Children’s Hospital, which will use Twitter data for monitoring of foodborne, gastrointestinal illness. Another health-related project will be carried out at the University of Twente in the Netherlands, where researchers will explore the diffusion and effectiveness of campaigns using Twitter to encourage early cancer detection. Japan’s National Institute of Information and Communications (NICT) aims to integrate its data from Twitter into a disaster information analysis system, and Australia’s University of Wollongong proposes an environmental study using geo-social intelligence to model urban flooding in the Indonesian capital, Jakarta. Finally, the University of East London in the U.K. will explore how tweets may affect or reflect the performance of sports teams.

The majority of scientists who study social media data today are from computer science. The team from the Software Studies Initiative is the only one of the winning teams with members who have strong backgrounds in arts, design, and humanities.

Yazdani, Manovich and Hochman also just had another achievement – a paper accepted to the 8^th International Conference on Weblogs and Social Media (ICWSM 2014), where it will be the only paper using ideas and methods from the humanities and media art, as opposed to computer science and social sciences, which are more typical for this conference. At the June 2014 ICWSM conference in Ann Arbor, Michigan, they will present their paper, “On Hyper-Locality: Performances of Place in Social Media.” The study explores large numbers of Instagram images documenting a month-long residency of the famous street artist Banksy in New York City during October 2013. For the paper, the Qualcomm Institute’s Yazdani did data analysis, including feature extraction, clustering, and classification.

As Lev Manovich put it on Friday, April 18: “It’s been a good week!”

It has also been a good year for Manovich and his colleagues on both coasts. They received extensive exposure for their Selfiecity.net project, which has now generated more than 110 original news articles in U.S. and international publicationsin the two months after the project was released in mid-February, including an article in New York Times. The project was also featured on the BBC, Discovery Channel and leading TV networks in a number of countries.

The #DataGrants award has already resulted in an interesting consequence for two of the winning teams. Manovich’s group was approached by the team at the Harvard Medical School to collaborate on a separate project that would involve the large dataset from Selfiecity.net. “The selfies and underlying data would be part of a global public health study,” said Manovich. “We are delighted to start working with them, because they are leaders in using social media for public-health research.”