Racing to the future

For most engineering classes, the final day is reserved for an exam. It’s a way for students to test their understanding of the key concepts by putting pen to paper, solving complex problems by applying the formulas they spent months committing to memory.

Rahul Mangharam in the School of Engineering and Applied Science has a different approach: “I like to test their skills by making them race.”

Not a foot race, he clarifies with a chuckle, but one with autonomous vehicles. In his class, students are graded on the design, assembly, and programming of self-driving cars that compete against each other in races.

These vehicles, which are about one-tenth of the size of a regular Formula 1 car—hence the name F1TENTH—are equipped with high-tech sensors and software the students need to code that allows the cars to navigate the track on their own at speeds of up to 40 miles per hour.

“You need to think of the cars as integrated complex systems that work together to make them move,” says Mangharam, a professor in the Departments of Electrical and Systems Engineering and Computer and Information Science. “I put the students into teams where different people work on different parts of the larger system, and then those teams compete to see who has the best performing car for their grade. It’s a lot of fun—we say it’s 10 times the fun at a tenth the size—but it’s also such a nice way for these budding engineers to truly put their solutions to the test.”

The students engage with a system that combines graphics processing units, Light Detection and Ranging (LIDAR), cameras, and other sensors, to teach the car to use that information to literally keep it on track.

The method to Mangharam’s madness

“This resonates a lot with the students because if they go out into the world and join a company, they are often in small groups testing out a little thing. They don’t see the whole picture,” Mangharam notes. The project’s educational model aims to bridge this gap by providing a platform where students can apply theoretical knowledge to practical challenges.

The course is for graduate students interested in robot perception, motion planning, control theory, and autonomous systems, Mangharam explains. It’s divided into lectures and extensive hands-on labs and designed so that students are introduced  to various facets of hardware, software, and the algorithms involved in building and racing a car.

By the sixth week, students will have built, programmed, and driven a tiny autonomous race car, and learned some fundamental principles of reactive planning, like how machines think on the fly for when things go awry.

By week 10, they have a firm grasp on simultaneous localization and mapping, which is how the cars will chart their surroundings while keeping track of where they are in space.

And by week 16, students will develop and implement advanced racing strategies that will give their team the edge in the race that concludes the course.

“Working on our F1TENTH car was stressful, but also just so exciting,” says Zhuolun (Alan) Zhao, a recent graduate of the robotics program. “You don’t know how fast the other teams’ cars are, so to beat them, you have to strike a balance between making yours fast but equally able to maneuver the course safely so it doesn’t bump into the walls or run off the track.”

Zhao’s team won one of the races, and notes that the team that won overall was the one that made the most of the open-source platform F1TENTH provides. He says the ones that spent the most time on optimizing their vehicles with mechanical upgrades and optimizing their code with simulations were the ones that saw better performances. 

“You’re not just learning about the theories behind autonomous systems; you’re seeing your code come to life in a real-world setting. It’s a huge leap from textbook learning.”

The genesis and evolution of F1TENTH

Since its inception at Penn Engineering in 2016, Mangharam has seen the F1TENTH project grow.

“Our first competition was a small event with just a few teams, but it was a huge success. The students loved it, and we saw a lot of enthusiasm and creativity,” Mangharam recalls. “From there, it just grew. We started getting interest from other universities, partners in industry, and before we knew it, we had a global competition on our hands.”

Today, F1TENTH is a global phenomenon with competitions held around the world. “We have had competitions in Japan, Korea, Hong Kong, Abu Dhabi, and Milan. These are all top conferences in robotics and transportation, and they want the F1TENTH competition to be part of their events because it’s so lively and exciting,” Mangharam says.

The open-source nature of their project, he explains, means that anyone can contribute and improve it. They have had more than 15 competitions this year, and will have six international competitions.

“Teams from ETH Zurich, Israel, Texas, Hungary, Poland, and Austria all build their vehicles based on our instructions from the F1TENTH website. They come and compete, and it’s fascinating to see the innovation and creativity from all over the world,” Mangharam says.

Another budding roboticist and former student of the class, Renu Reddy, notes the value of the collaborative nature of this  project. “Everybody thinks differently, brings their own perspective, and being able to work with other students on your team is great because solving a problem is just about how you think. A little tweak here and there can help you move in the right direction,” she says.

Reddy says the experience with the F1TENTH car project provided her with the foundational skills to tackle another ambitious project in Mangharam’s lab: the autonomous go-kart, known as AV4EV. This go-kart is a one-third scale autonomous vehicle platform based on a commercial go-kart chassis, which supports a flexible range of sensing and computing and can carry a human driver.

“Our go-kart platform bridges the gap between RC cars and full-scale vehicles, providing an effective solution for autonomous vehicle research,” Mangharam says.

The project involves designing and implementing a full autonomy stack, which includes perception, planning, and control algorithms that enable the go-kart to safely and efficiently navigate complex environments.

Reddy’s role in the go-kart project focused on integrating the perception system, “Right now, I’m working on integrating the camera data with the LIDAR data and getting high-definition maps of the surroundings so that we know what is around us,” she explains.

View large image

Her work on localization techniques, specifically adaptive localization methods that use random sampling and statistical analysis to estimate the position of the vehicle, which is known as the Monte Carlo method, allows the go-kart to accurately figure out its position within these maps and continuously refines its position estimates by comparing predicted sensor readings with the actual readings.

“Once we know what is around us and where we are, the next step for any autonomous system is to be able to plan a path to the goal,” Reddy says. “We’re working on planning algorithms that track the go kart’s path and adjust its steering and to get an optimal route without colliding into obstacles, much like how we did with the F1TENTH vehicle, but just bigger.”

Building bigger, better, safer autonomous systems

Mangharam’s goal is to further develop the open-source platforms to enable more universities and research institutions to participate and contribute to projects like F1TENTH and AV4EV.

“We need to democratize access to advanced autonomous vehicle technology, making it possible for anyone with the interest and drive to get involved,” he says, adding that he believes this work is essential for a future where driverless cars become a reality.

“Autonomous vehicles represent the next frontier in transportation, with the potential to drastically reduce accidents, improve traffic flow, and enhance mobility for all,” he says. “However, achieving this vision requires disseminating mission-critical information to address any and every safety concern. Safety is paramount. We need to be able to trust these systems with our lives, and that requires rigorous testing and validation.”

Another goal is to enhance the educational framework surrounding these projects. Mangharam plans to integrate more interdisciplinary learning opportunities, involving students from various engineering fields as well as disciplines like psychology and ethics.

“Understanding the human factors and ethical implications of autonomous systems is just as crucial as the technical components,” he explains. “This holistic approach aims to produce well-rounded engineers capable of addressing the multifaceted challenges of autonomous vehicle technology. Our goal is to equip them with the tools and mindset to tackle the challenges and opportunities of tomorrow.”