EDAG Worldwide

    Vehicle Engineering
    br
    Brazil
    cn
    China
    gb
    Great Britain
    it
    Italy
    jp
    Japan
    my
    Malaysia
    mx
    Mexico
    nl
    Netherlands
    se
    Sweden
    pl
    Poland
    ch
    Switzerland
    es
    Spain
    cz
    Czech Republic
    hu
    Hungary
    us
    USA
    Production Solutions
    br
    Brazil
    cn
    China
    in
    India
    mx
    Mexico
    se
    Sweden
    cz
    Czech Republic
    hu
    Hungary
    us
    USA

    tech insights

    An AI based raindrop detection system for Autonomous Driving

    On one hand automotive industry’s largest researched field, Autonomous Driving is leading to revolutionary shift by teaching the cars how to drive on its own in both controlled and uncontrolled environments. On the other hand, it is almost impossible to develop cars that can surpass human driving without a powerful and a reliable sensor system, especially when driving in adverse climatic conditions such as rain and snow.

    A camera sensor has become a fundamental and the most important part of the sensor set system used in Autonomous cars. It is through a camera, a car can visually understand what is around it, something identical to what eyes do to humans. Hence, its reliability is key for safer and better driving.

    Existing driver assistance systems work exceptionally well in clear weather conditions. However, it is unavoidable that when driving through adverse climatic conditions such as rain or snowfall, contaminants such as raindrop, snowflakes, dirt or soil adhere to the lens of the camera and the background visibility is hampered severely. Especially when raindrops adhere to the lens of the camera, the background is blurred and there is either loss of information or misinterpretation of data. Consequently, this may lead to the failure of various driver assistance features such as pedestrian detection, object detection, traffic sign detection and many others. Thus, it has to be made sure that the camera delivers clear and complete information for better and safe driving.

    textbild-1-RDNet

    Figure 1: Image captured from a camera with raindrops adhered on its lens. It can be observed how the background visibility is hampered and blurred.

    A solution to avoid the above-stated problem is to ensure that there should be no compromise in the quality of the image received from a camera sensor. This can be achieved by developing a software to post process images received from the camera and reconstruct the images if they are deformed. However, in order to regain the information lost, these regions have to be first detected in the deformed images. At EDAG Electronics, a software based on Artificial Intelligence is developed that can detect adherent raindrops on single images. A software solution to the above stated problem is a straightforward and provides a permanent and a reliable solution. Additionally, it is hardware independent and a single software can be used in multiple cameras placed in a car (i.e., front-view, top-view, rear-view, surround-view cameras).

    The raindrop detection system developed is called as RDNet (Raindrop Detection Network). It takes in an image containing adherent raindrops, processes the image, and generates a binary mask containing precise location information of the adherent raindrops in the images. The raindrops are segmented into white regions and the non-raindrop (background) regions are segmented as black.

    textbild-2-RDNet

    Figure 2: A pictorial illustration of the raindrop detection system (RDNet). Images from left: Input image with raindrops adhered on it, middle: Model, right: Binary mask generated by the model that contains precise raindrop location information.

    RDNet – The Raindrop Detection Network

    The RDNet developed to detect raindrops is a type of neural network called as Convolution Neural Network (CNN) that is specially designed to work with images. A CNN can be trained to identify and learn the features of a raindrop precisely, rather than using the conventional feature description techniques. Owing to significant variations in the properties of raindrops on the ground of its size, shape and texture, simple image segmentation techniques based on traditional feature description methods are not sufficient to produce accurate results for a complex use case such as Autonomous Driving.

    A CNN declines the traditional programming pattern and rather finds its basis on learning from examples. Instead of manually defining the features of a raindrop one by one and forming an algorithm, a few convolution layers put together forming a neural network can be trained on a raindrop dataset and the network learns the features of the raindrops and the features of the continuously varying background on its own.

    A CNN is a special form of supervised learning method where the network learns from a pair of inputs. The training data consists of both, the input images with raindrops and the desired output also known as binary labels. The network when trained on such a dataset learns to associate the input image with the desired output (labels). Once the network is trained, the model gains the ability to detect and predict raindrops on unseen examples based on its learning.

    The biggest challenge in any deep learning task is the data availability. To train the RDNet, a special raindrop dataset was used where the raindrops are generated synthetically. The dataset contains 35700 single training images and corresponding binary labels. In addition, to make it more realistic and practical, the background of these images are of urban areas under cloudy weather conditions.

    textbild-3-RDNet

    Figure 3 : Sample of the image with adherent raindrops used to train the network. The raindrops are synthetically generated on this background.

    By integrating the RDNet with a reconstruction software such as the DiFoRem (another in-house software developed to reconstruct portions of images occluded by contaminants), the deformed regions of the images can be reconstructed. The DiFoRem uses the binary masks generated by the RDNet, to produce clean and clear images with improved background visibility. The RDNet will be further expanded in the future to also detect other forms of contaminants such as dirt, snow, soil etc. The detections made using the RDNet are shown below:

    textbild-4-RDNet

    Figure 4: The detections made by using the RDNet. Images from top: Raindrop image, middle: The predicted mask and bottom: Raindrops detected by the network are highlighted in blue.

    AI has paved way for turning Autonomous cars that was once only a concept now into reality. It enables the vehicle to perceive, hear and make decisions just like a human. At EDAG, we believe in “Always a little different and always a little ahead of time”. Thus, the solutions to define and develop vehicle technologies that ensure better and safer driving are always future oriented. We are happy to accompany our customers with futuristic solutions in making Autonomous Driving a reality.

    Our team in Lindau / Ulm is working intensively on tasks in the areas of Deep Learning and Computer Vision. If you are interested in these technologies or in other application areas of AI, Malavika Venugopal, AI software developer, will be happy to help you.

    For more information, download the white paper "How a combination of AI and camera systems clears the way for the future of autonomous driving".

    White paper Raindrop Detection Network

    Download white paper now
    Schedule  an Expert  Call >>

    Related Post

    The fully autonomous passenger car is still a long way off. However, driver assistance systems that can at least provide support are advancing at a rapid rate. Essentially, they are based on recognition of the vehicle's environment. Artificial intelligence helps ensure that they work reliably even in adverse weather conditions and if there is dirt...

    >> Read more
    The question of when a component or subsystem will reach the end of its life cycle and be in danger of failing is even more critical in aerospace engineering than in other industries. It is true that more and more sensors are monitoring the loads and functionality of numerous components. However, this flood of data calls for modern software...

    >> Read more
    When designing components, conserving energy and material is now an obligation. Using computer-based simulation procedures makes it possible to configure the shape of a component for minimal material use – this is referred to as topology optimization. If there are errors in this process, then there will be failures in practical application, if the...

    >> Read more
    EDAG Logo

    EDAG

    Kreuzberger Ring 40, 65205 Wiesbaden
    p +49 661 6000-0 f +49 661 6000-223