Sherlock Deploys UC’s first HIPAA-compliant Hadoop based Data Management System

Article Content

The Health Cyberinfrastructure Division at the San Diego Supercomputer Center (SDSC) at the University of California San Diego, in partnership with University of California Office of the President (UCOP) Risk Services, has deployed a secure, HIPAA-compliant Hadoop-based Data Management System (DMS) within Sherlock Cloud.

This secure platform will provide a mechanism to collect, process, and transform UC Healthcare data from multiple sources and various formats into a single, integrated data set to enable UCOP Risk Services better manage UC’s liability program.

SDSC’s Health CI Division will support the UCOP Risk Services and UC Medical Centers initiative to deliver a consolidated data warehouse. This platform will facilitate access to data, reports, and advanced analytics, allowing for more informed decision-making to support quality and risk improvement programs in place throughout the UC Medical Centers. Not only will DMS improve how the data is collected, stored, and analyzed, but it will also mitigate claims costs and ensure the highest quality of care within the UC Medical Centers system-wide.   

When conceptualizing the DMS, the team decided on an alternative approach to the traditional data warehouse and opted to apply the “Big Data” paradigm to this system.  In doing so, the DMS infrastructure utilizes Cloudera’s Hadoop technology, which provides several benefits such as:

• Allowing raw data to be stored and transformed into a consumable form without the need to design a schema;
• Addressing data volume by scaling out as opposed to scaling up, which helps to preserve the efficiency with which data is processed and maintains costs;
• Facilitating storage of unstructured data.

“This platform presents an opportunity to demonstrate how best-of-breed Big Data technology solutions can be used to support large-scale data management and analytics for the varied types of protected data managed by UC, all operating within the secure boundaries of Sherlock Cloud,” said Sandeep Chandra, Executive Director of Sherlock Cloud. “The valuable experience we have gained in standing up this unique platform presents an opportunity for us to offer such capability to others in the research community. This initiative naturally builds on the successful partnership SDSC has had with UCOP Risk Services in supporting many of their business and technology needs, and we are excited to see it grow.”