Students test and train their autonomous vehicles on a course they made around the Engineering Building II courtyard.

Undergraduate Engineers Get Hands-on Experience with Autonomous Vehicles

It’s one thing to learn about computer vision, machine learning, control systems, neural networks and tools like Python and TensorFlow in lectures, books or videos. It’s another to learn these skills by actually using them to build and test small autonomous vehicles. That’s what students are doing in a hands-on engineering class at the Jacobs School of Engineering at the University of California San Diego.

“The whole point for this class is that we’re trying to build a robot that, between the perception and computer system, is about $80,” said Jack Silberman, one of two instructors of the Introduction to Autonomous Vehicles course run jointly by the Departments of Mechanical and Aerospace Engineering and Electrical and Computer Engineering at the UC San Diego Jacobs School of Engineering. He’s joined by Mauricio de Oliveira, an adjunct professor in the Department of Mechanical and Aerospace Engineering.

“We design a vehicle that’s very industry relevant. A lot of the industry keep talking about deep learning, they keep talking about hands-on learning, TensorFlow, Keras—those are tools that, if the students graduate today, they will be able to apply for a job based on that skill,” Silberman said.

The course is all about hands-on learning by doing. Over the course of the quarter, undergraduate and master’s students enrolled in Introduction to Autonomous Vehicles build a small robotic car, train it to run autonomously, and trick it out with a bonus feature of their choosing.

At the start of the class, students break into groups of four and spend the first week or two building their vehicle using Raspberry Pi and off-the-shelf hardware and electrical components, and programming it to move, turn and stop using Python.

Then the fun begins.

Each team gets a budget of $100 to enhance their car in any way they choose. One team during the summer session purchased an ultrasonic sensor and laser to see which is more effective for auto-stop capability. Another bought cameras for 3D stereo vision. A third team designed an emergency braking system, while another designed an object recognition system.

Silberman and de Oliveira run the class like a company, so teams can barter with each other for the rights to any surplus budget: if one team only spends $80, another team wanting a more expensive piece of equipment can work with them to claim the unspent funds.

An affordable platform for students to learn the skills required to actually make an autonomous vehicle.

Simultaneously, the students are remote controlling their cars around the track time and time again—at least 50 laps—and using all of that data to train a neural network so the cars can maneuver around the course on their own. Completely autonomously.

Seeing the results of their efforts was a motivating factor for some students, but many said that the real benefit was the interdisciplinary nature of a hands-on, project-based class like this.

“I’ve never really had experience using Linux or Python and so it’s one of those things where it’s good to have team members who are in electrical engineering because they know what they’re doing with that and you can shadow what they’re doing,” said Albert Hernandez, a third year mechanical and aerospace engineering student who said he’d consider pursuing a career in robotics or autonomous vehicles.

Hernandez said he has a solid understanding of how to design and make things, but was motivated to take this class to learn more of the electrical engineering principles and skills that are required to get parts to move in a coordinated, precise way.

His teammate Jianxin “Cathy” Tran, also a third year mechanical engineering student, said the course gave her a basic understanding of computer science that made her feel more comfortable taking classes in that discipline.

“I think this class encouraged me to take more computer science classes for sure-- I wasn’t really afraid of computer science, but right now I feel like I definitely need to take it,” Tan said. “And since mechanical engineering is in some ways a classical major, I think it’s important to combine that with new technologies, which this class does.”

That was one of the goals of the class, said de Oliveira— for students to work with partners from a variety of engineering backgrounds.

“This is a great opportunity for the students to work on a multi-disciplinary team,” de Oliveira said. “Even though there are many hands-on classes in particular departments, we make sure we put together teams with electrical engineers, computer scientists, mechanical and aerospace engineers, all working on this one project together.”

Small car, big data

Neural networks and handling large quantities of data were also new to most students in the Introduction to Autonomous Vehicles class. When they’re remote controlling their vehicles around the track, a camera on the car is capturing images and video of the route. All of this imagery needs to be uploaded to a computer and run through a machine learning algorithm so the car can learn to navigate, but students don’t typically have the computing power necessary to store and process that much data.

Enter a collaborative effort between IT Services, the San Diego Supercomputer Center, and the Qualcomm Institute. Thanks to the Data Science Machine Learning Platform—utilizing a Qualcomm Institute design, hosted at the Supercomputer Center and run by UC San Diego’s Information Technology Services—the students are able to remotely upload their data to the Supercomputer, where it’s run through machine learning algorithms, and then downloaded back into the car, allowing the car to navigate autonomously.

“We were looking at where data science and machine learning are going on campus and realized we needed a central resource,” said Adam Tilghman, who manages the platform. “Our role is to support undergraduate and graduate classroom instruction, and to some degree thesis research for graduate students as well.”

The Data Science Machine Learning Platform launched in the fall of 2017, and served more than 2,500 students in its first year, including those from the Autonomous Vehicle class. Students across campus are able to store and process data on the Platform’s 96 high-end Graphics Processing Units thanks to Tilghman and his team allocating use on the backend.

Tilghman said in addition to the hardware and software skills the students learn, knowing how to handle data and train neural networks are also very valuable skills.

“They have these data sets that come out of their series of photo frames coming off the car that can be a fairly large assemblage of data—tens of thousands or hundreds of thousands of snapshots. Working with data sets that large is something that employers I think would respect and see value in.”

Future Plans

The autonomous vehicle course is part of the Jacobs School of Engineering’s Experience Engineering Initiative to ensure every engineering and computer science undergraduate has a hands-on or experiential engineering course every year.

“Early on, we must give our students many opportunities for “a-ha!” moments — experiences that help students generate the confidence and motivation they need to persevere and succeed in classes, internships and ultimately in engineering careers,” said Albert P. Pisano, Dean of the Jacobs School of Engineering. “From personal teaching experience, I know that these kinds of hands-on classes propel undergraduates to higher levels of excellence in their coursework, research and internships.”

The intent of the Autonomous Vehicle course isn’t to prepare students to enter the autonomous vehicle field necessarily, though some students say they are considering that.

“I’m actually kind of considering such paths, because I’m doing a project about simulations, localization and mapping which is heavily connected to autonomy,” said Lu Xuanchen, a second year electrical engineering student. “I think it’s pretty fun to implement and design algorithms as well as hardware to help the robot to understand the world.”

The intent is to provide a relevant, hands-on way of learning the skills necessary to have an impact in whatever venue students choose to pursue after graduation.

To that end, Silberman, a lecturer in the Jacobs School of Engineering and the commercial quality assurance manager for Canada and Latin America with Abbott Diabetes Care, said the class is run the same way he runs his industry teams.

“It is 100 percent how actually I run my engineers in the industry,” he said. “They have to work together. And then the evaluation is between how I did something, not just what I did.

“They have a chance to design something, to prove it to work, and if it doesn’t work explain why it didn’t work. And they get the fundamentals of project management, definitely do a lot of team working, and they learn how to manage a budget also.”