Bioengineers Researching Smart Cameras and Sensors that Mimic, Exceed Human Capability

Article Content

Cauwenberghs' laboratory developed a neuromorphic circuit array that models computation and communication across large-scale networks in the visual cortex in order to understand how the brain receives and processes visual information. Each chip in the array mimics the activity of 65 thousand neurons that make 65 million synaptic connections in memory. Photo Credit: UC San Diego Jacobs School of Engineering.

University of California, San Diego bioengineering professor Gert Cauwenberghs has been selected by the National Science Foundation to take part in a five-year, multi-institutional, $10 million research project to develop a computer vision system that will approach or exceed the capabilities and efficiencies of human vision. The Visual Cortex on Silicon project, funded through NSF’s Expeditions in Computing program, aims to create computers that not only record images but also understand visual content and situational context in the way humans do, at up to a thousand times the efficiency of current technologies, according to an NSF announcement.

Smart machine vision systems that understand and interact with their environments could have a profound impact on society, including aids for visually impaired persons, driver assistance capabilities for reducing automotive accidents, and augmented reality systems for enhanced shopping, travel, and safety.

For their part in the effort, Cauwenberghs, a professor in the Department of Bioengineering at the UC San Diego Jacobs School of Engineering, and his team are developing computer chips that emulate how the brain processes visual information. “The brain is the gold standard for computing,” said Cauwenberghs, adding that computers work completely differently than the brain, acting as passive processors of information and problems using sequential logic. The human brain, by comparison, processes information by sorting through complex input from the world and extracting knowledge without direction.

While several computer vision systems today can each successfully perform one or a few human tasks-such as detecting human faces in point-and-shoot cameras-they are still limited in their ability to perform a wide range of visual tasks, to operate in complex, cluttered environments, and to provide reasoning for their decisions. In contrast, the visual cortex in mammals excels in a broad variety of goal-oriented cognitive tasks, and is at least three orders of magnitude more energy efficient than customized state-of-the-art machine vision systems.

Cauwenberghs and his collaborators around the country aim to understand the fundamental mechanisms used in the visual cortex, with the hope of enabling the design of new vision algorithms and hardware fabrics that can improve power, speed, flexibility and recognition accuracies relative to existing computer vision systems.

Ph.D. graduate student Siddharth Joshi at work designing a computer chip that mimics brain function.

Cauwenberghs said the Visual Cortex on Silicon project offers a unique collaborative opportunity with experts across the globe in neuroscience, computer science, nanoengineering and physics.

The project has other far-reaching implications for neuroscience research. By developing chips that can function more like the human brain, Cauwenberghs believes researchers can achieve a number of significant breakthroughs in our understanding of brain function from the work of single neurons all the way up to a more holistic view of the brain as a system. For example, building chips that model different aspects of brain function, such as how the brain processes visual information, gives researchers a more robust tool to understand where problems arise that contribute to disease or neurological disorders.

The Expeditions in Computing program, which started in 2008, represents NSF’s largest single investments in computer science research. As of today, 16 awards have been made through this program, addressing subjects ranging from foundational research in computing hardware, software and verification to research in sustainable energy, health information technology, robotics, mobile computing, and Big Data.