All About Self-Driving Cars

Did you know – the first self-driving car was invented in 1989, called the Automatic Land Vehicle in Neural Network by Carnegie Mellon University? It used deep learning methods to understand its environment and navigate, but it was limited by compute power and limited data.

Today, the potential is immense for self-driving cars, with availability of high-performance graphics cards, processors and “big data”. Google's Waymo is building the world’s first autonomous ride-hailing service and trucking solutions. In 2015, BMW i3 parked itself in the garage by recognizing structural features of the garage and navigating around unexpected objects. Aurrigo is expected to roll out autonomous buses in England for public use soon. But how does this technology actually work?

What is a Self-Driving Car?

A self-driving car is an autonomous vehicle that senses its environment and operates with minimal (but present) human control, leveraging sensors, actuators, ML systems and high-performance processors. It can prove useful in reducing traffic congestion and increasing road safety.

There are 6 stages of driving automation defined by Society of Automotive Engineers (SAE) –

1. No Automation (complete human control)
2. Driver Assistance (single automated feature, e.g., cruise control)
3. Partial Automation (advanced driver assistance system – ADAS)
4. Conditional Automation (machine can perform most driving tasks)
5. High Automation (machine performs all driving tasks under specific conditions)
6. Full Automation (machine performs all driving tasks under all conditions)

A fully autonomous car is self-aware and can make its own choices. This is why, when we refer to a self-driving car, we don’t refer to it strictly as an autonomous car, because it still requires human control.

How does a Self-Driving Car work?

Let’s see how a self-driving cars works in 2 high-level steps:

1.Information about the environment is gathered from below sources -

• Radar sensors (monitor position of nearby vehicles)
• Video cameras (visual capture of environment)
• Light detection and ranging (LiDAR) for distance measurement, road edge detection etc.
• Ultrasonic sensors for detecting curbs

2. The sensory information is passed to Machine Learning systems comprising of computer vision algorithms and reinforcement learning, that decide navigation based on the environment and send instruction to actuators that control acceleration, braking and steering. This is not as easy as it sounds, because the tasks involve -

• Perception: Seeing environment and recognizing objects through classification, segmentation and localization
• Localization: Calculating position and orientation of vehicle as it navigates, also called Visual Odometry
• Prediction: Predicting trajectory of human drivers/pedestrians to prepare for all possible moves
• Decision making: Making decision about best action to take from set of all possible actions, through deep reinforcement learning

Let’s dive further into key computer vision and reinforcement learning technologies used by self-driving cars today.

Technologies for Self-Driving Cars

There are different architectures of computer vision algorithms being used by different self-driving car companies. Some examples are:

• HydraNet by Tesla: Originally developed for semantic segmentation, it has a dynamic architecture that helps it adapt to different environments and different tasks
• Nvidia self-driving car: Uses 3 cameras (right, centre and left) and convolutional neural networks for navigation
• ChauffeurNet by Waymo: Convolutional neural networks are used for feature extraction, that are then fed to a Recurrent Neural Network (AgentRNN) that predicts driving trajectory

The other part of the story is regarding decision-making, where reinforcement learning comes in. Reinforcement learning is a type of machine learning where an agent learns by interacting with the environment, and is based on an state-action-reward system unlike supervised learning. Some common open-source simulators are:

• CARLA
• SUMMIT
• AirSim by Microsoft Research
• DeepDrive
• Flow

Some other development platforms for reinforcement learning are:

• AWS Deep Racer
• Deep Traffic by MIT

The Future

There is a lot of promise in self-driving cars, but human intervention is currently necessary since some roads lack signs and markings, the ML systems need to become more accurate, and vehicle-to-vehicle communication is yet to be developed. The dream is to reach the sixth level of driving automation!