About Self Driving Car Challenge TAB

Engineering students entering the Industry will need to be equipped with practical knowledge and design experience.

Challenge Objective

  • Detect highway lane lines on a video stream.
  • Use OpenCV image analysis techniques to identify lines, including Hough Transforms and Canny edge detection.
  • Build and train a deep neural network to classify traffic signs, using TensorFlow.
  • Experiment with different network architectures.
  • Perform image pre-processing and validation to guard against overfitting.
  • Build and train a convolutional neural network for end-to-end driving in a simulator, using TensorFlow and Keras.
  • Use optimization techniques such as regularization and dropout to generalize the network for driving on multiple tracks.
  • Build an advanced lane-finding algorithm using distortion correction, image rectification, color transforms, and gradient thresholding.
  • Identify lane curvature and vehicle displacement.
  • Overcome environmental challenges such as shadows and pavement changes.
  • Create a vehicle detection and tracking pipeline with OpenCV, histogram of oriented gradients (HOG), and support vector machines (SVM).
  • Implement the same pipeline using a deep network to perform detection.
  • Optimize and evaluate the model on video data from an automotive camera taken during highway driving.
  • Implement the extended Kalman filter in C++.
  • Simulate lidar and radar measurements are used to detect a bicycle that travels around your vehicle.
  • Use Kalman filter, lidar measurements and radar measurements to track the bicycle’s position and velocity.

Challenge Elements 

The following 13 Steps & Projects have to be completed for Self-Driving Car Challenge.

Project Id Project Name
1 Lane Finding Basic
2 Traffic Sign Classifier
3 Behavioral Cloning
4 Advanced Lane Finding
5 Vehicle Detection
6 Extended Kalman Filter
7 Unscented Kalman Filter
8 Kidnapped Vehicle
9 PID Control
10 MPC Control
11 Path Planning
12 Road Segmentation
13 Traffic Light Classifier
  • Lane Finding Basic:The goal of this first project was to create a simple pipeline to detect road lines in a frame taken from a roof-mounted camera.
  • Traffic Sign Classifier:The goal of this project was to build a CNN in TensorFlow to classify traffic sign images from the Traffic Sign Dataset.
  • Behavioral Cloning:The goal of the project was to train a Deep Network to replicate the human steering behavior while driving, thus being able to drive autonomously on a simulator provided by SAEISS. To this purpose, the network takes as input the frame of the frontal camera (say, a roof-mounted camera) and predicts the steering direction at each instant.
  • Advanced Lane Finding: The goals / steps of this project are the following:
    • Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
    • Apply a distortion correction to raw images.
    • Use color transforms, gradients, etc., to create a thresholded binary image.
    • Apply a perspective transform to rectify binary image (“birds-eye view”).
    • Detect lane pixels and fit to find the lane boundary.
    • Determine the curvature of the lane and vehicle position with respect to center.
    • Warp the detected lane boundaries back onto the original image.
    • Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
  • Vehicle Detection: The goal of the project was to develop a pipeline to reliably detect cars given a video from a roof-mounted camera.
  • Extended Kalman Filter: This goal is to implement the extended Kalman filter in C++. We are providing simulated lidar and radar measurements detecting a bicycle that travels around your vehicle. You will use a Kalman filter, lidar measurements and radar measurements to track the bicycle’s position and velocity.
  • Unscented Kalman Filter: The goals / steps of this project are the following:
    • The overall processing chain (prediction, laser update or radar update depending on measurement type) must be correct.
    • The student is not allowed to use values from the future to reason about the current state.
    • It must be possible to run the project in three different modes: considering laser only, with considering radar only, or with using both sensors.
    • For every mode, the overall RMSE (2d position only) may not be more than 10% increased to what the original solution is able to reach (this number depends on the individual measurement sequence)
    • The RMSE of laser AND radar must be lower than radar only or laser only
    • The NIS of radar measurements must be between 0.35 and 7.81 in at least 80% of all radar update steps.
  • Kidnapped Vehicle: The goals / steps of this project are the following:
    • Your vehicle has been kidnapped and transported to a new location! Luckily it has a map of this location, a (noisy) GPS estimate of its initial location, and lots of (noisy) sensor and control data.
    • In this project you will implement a 2-dimensional particle filter in C++. Your particle filter will be given a map and some initial localization information (analogous to what a GPS would provide). At each time step your filter will also get observation and control data.
  • PID Control: The goal of this project is to implement a PID controller for keeping the car on track by appropriately adjusting the steering angle.
  •  MPC Control: The goal of this project is to implement the cost function in PID control.
  • Path Planning: The goal of this project is to safely navigate around a virtual highway with other traffic that is driving +-10 MPH of the 50 MPH speed limit. You will be provided the car’s localization and sensor fusion data, there is also a sparse map list of waypoints around the highway. The car should try to go as close as possible to the 50 MPH speed limit, which means passing slower traffic, when possible, note that other cars will try to change lanes too. The car should avoid hitting other cars at all cost as well as driving inside of the marked road lanes at all times, unless going from one lane to another. The car should be able to make one complete loop around the 6946m highway. Since the car is trying to go 50 MPH, it should take a little over 5 minutes to complete 1 loop. Also, the car should not experience total acceleration over 10 m/s^2 and jerk that is greater than 50 m/s^3.
  • Road Segmentation: The goal of this project, is to label the pixels of a road in images using a Fully Convolutional Network (FCN).
  • Traffic Light Classifier: The goal of this project is to integrate Traffic light classifier.