EDAG Worldwide

    Vehicle Engineering
    br
    Brazil
    cn
    China
    gb
    Great Britain
    it
    Italy
    jp
    Japan
    my
    Malaysia
    mx
    Mexico
    nl
    Netherlands
    se
    Sweden
    pl
    Poland
    ch
    Switzerland
    es
    Spain
    cz
    Czech Republic
    hu
    Hungary
    us
    USA
    Production Solutions
    br
    Brazil
    cn
    China
    in
    India
    mx
    Mexico
    se
    Sweden
    cz
    Czech Republic
    hu
    Hungary
    us
    USA

    tech insights

    An AI based raindrop detection system for Autonomous Driving

    On one hand automotive industry’s largest researched field, Autonomous Driving is leading to revolutionary shift by teaching the cars how to drive on its own in both controlled and uncontrolled environments. On the other hand, it is almost impossible to develop cars that can surpass human driving without a powerful and a reliable sensor system, especially when driving in adverse climatic conditions such as rain and snow.

    A camera sensor has become a fundamental and the most important part of the sensor set system used in Autonomous cars. It is through a camera, a car can visually understand what is around it, something identical to what eyes do to humans. Hence, its reliability is key for safer and better driving.

    Existing driver assistance systems work exceptionally well in clear weather conditions. However, it is unavoidable that when driving through adverse climatic conditions such as rain or snowfall, contaminants such as raindrop, snowflakes, dirt or soil adhere to the lens of the camera and the background visibility is hampered severely. Especially when raindrops adhere to the lens of the camera, the background is blurred and there is either loss of information or misinterpretation of data. Consequently, this may lead to the failure of various driver assistance features such as pedestrian detection, object detection, traffic sign detection and many others. Thus, it has to be made sure that the camera delivers clear and complete information for better and safe driving.

    textbild-1-RDNet

    Figure 1: Image captured from a camera with raindrops adhered on its lens. It can be observed how the background visibility is hampered and blurred.

    A solution to avoid the above-stated problem is to ensure that there should be no compromise in the quality of the image received from a camera sensor. This can be achieved by developing a software to post process images received from the camera and reconstruct the images if they are deformed. However, in order to regain the information lost, these regions have to be first detected in the deformed images. At EDAG Electronics, a software based on Artificial Intelligence is developed that can detect adherent raindrops on single images. A software solution to the above stated problem is a straightforward and provides a permanent and a reliable solution. Additionally, it is hardware independent and a single software can be used in multiple cameras placed in a car (i.e., front-view, top-view, rear-view, surround-view cameras).

    The raindrop detection system developed is called as RDNet (Raindrop Detection Network). It takes in an image containing adherent raindrops, processes the image, and generates a binary mask containing precise location information of the adherent raindrops in the images. The raindrops are segmented into white regions and the non-raindrop (background) regions are segmented as black.

    textbild-2-RDNet

    Figure 2: A pictorial illustration of the raindrop detection system (RDNet). Images from left: Input image with raindrops adhered on it, middle: Model, right: Binary mask generated by the model that contains precise raindrop location information.

    RDNet – The Raindrop Detection Network

    The RDNet developed to detect raindrops is a type of neural network called as Convolution Neural Network (CNN) that is specially designed to work with images. A CNN can be trained to identify and learn the features of a raindrop precisely, rather than using the conventional feature description techniques. Owing to significant variations in the properties of raindrops on the ground of its size, shape and texture, simple image segmentation techniques based on traditional feature description methods are not sufficient to produce accurate results for a complex use case such as Autonomous Driving.

    A CNN declines the traditional programming pattern and rather finds its basis on learning from examples. Instead of manually defining the features of a raindrop one by one and forming an algorithm, a few convolution layers put together forming a neural network can be trained on a raindrop dataset and the network learns the features of the raindrops and the features of the continuously varying background on its own.

    A CNN is a special form of supervised learning method where the network learns from a pair of inputs. The training data consists of both, the input images with raindrops and the desired output also known as binary labels. The network when trained on such a dataset learns to associate the input image with the desired output (labels). Once the network is trained, the model gains the ability to detect and predict raindrops on unseen examples based on its learning.

    The biggest challenge in any deep learning task is the data availability. To train the RDNet, a special raindrop dataset was used where the raindrops are generated synthetically. The dataset contains 35700 single training images and corresponding binary labels. In addition, to make it more realistic and practical, the background of these images are of urban areas under cloudy weather conditions.

    textbild-3-RDNet

    Figure 3 : Sample of the image with adherent raindrops used to train the network. The raindrops are synthetically generated on this background.

    By integrating the RDNet with a reconstruction software such as the DiFoRem (another in-house software developed to reconstruct portions of images occluded by contaminants), the deformed regions of the images can be reconstructed. The DiFoRem uses the binary masks generated by the RDNet, to produce clean and clear images with improved background visibility. The RDNet will be further expanded in the future to also detect other forms of contaminants such as dirt, snow, soil etc. The detections made using the RDNet are shown below:

    textbild-4-RDNet

    Figure 4: The detections made by using the RDNet. Images from top: Raindrop image, middle: The predicted mask and bottom: Raindrops detected by the network are highlighted in blue.

    AI has paved way for turning Autonomous cars that was once only a concept now into reality. It enables the vehicle to perceive, hear and make decisions just like a human. At EDAG, we believe in “Always a little different and always a little ahead of time”. Thus, the solutions to define and develop vehicle technologies that ensure better and safer driving are always future oriented. We are happy to accompany our customers with futuristic solutions in making Autonomous Driving a reality.

    Our team in Lindau / Ulm is working intensively on tasks in the areas of Deep Learning and Computer Vision. If you are interested in these technologies or in other application areas of AI, Malavika Venugopal, AI software developer, will be happy to help you.

    For more information, download the white paper "How a combination of AI and camera systems clears the way for the future of autonomous driving".

    White paper Raindrop Detection Network

    Download white paper now
    Schedule  an Expert  Call >>

    Related Post

    With the IAA Mobility 2021 in Munich at the beginning of September, the rapid changes in the automotive industry and the recent dramatic climate developments, the mobility of tomorrow is more in the public focus than ever before. New technologies, alternative drives, smart city and power grid concepts, the networking of all vehicles and autonomous...

    >> Read more
    Vehicles that navigate their way through everyday traffic on their own, while the driver works on a laptop or relaxes in the back seat: These images are still dreams of the future. Even if no fully autonomous vehicle has yet been approved for use on the roads, car manufacturers are increasingly installing driver assistance systems as standard....

    >> Read more
    In 1927, Charles Lindbergh flew his plane "Spirit of St. Louis" the 5,800 kilometres from New York to Paris in 33 long and lonely hours. Not quite 50 years later, Concorde covered the same distance in an incredible three supersonic hours. Flying is something we take for granted today, and we seldom take the time to appreciate the extremely complex...

    >> Read more
    EDAG Logo

    EDAG

    Kreuzberger Ring 40, 65205 Wiesbaden
    p +49 661 6000-0 f +49 661 6000-223