Trained neural networks should ensure that autonomous vehicles always react appropriately
Trained neural networks should ensure that autonomous vehicles always react appropriately
( Source: Fraunhofer Institut)

Autonomous Mobility How Neuromorphic Technology Improves Environment Assessment

Editor: Florian Richert

Scientists want to develop a new AI platform for autonomous driving. The programmable, ASIC-based control hardware, will use neuromorphic technology to respond flexibly and independently to any changes in driving conditions.

The Fraunhofer Institute for Integrated Circuits IIS, together with partners in the "KI-FLEX" joint project, is developing a platform that uses methods of artificial intelligence to help determine the exact position and environment of the vehicle. The platform uses several sensors, weighted differently depending on the currently prevailing environmental conditions. The system should enable fully automated and autonomous vehicles to react adequately in any situation.

Inspired by the brain, designed for autonomous mobility

KI-FLEX is a software programmable and reconfigurable hardware platform for AI-based sensor data processing for autonomous driving. The project should make an essential contribution to the development of urgently needed technology components that make autonomous driving safe and reliable.
In autonomous mobility, data from a laser, camera, and radar sensors in cars are processed and merged reliably and quickly. The systems are the prerequisite for the vehicle to have a precise image of the real traffic situation at all times. It enables the vehicle to locate itself in this environment and to make the right decision in every driving situation. However, the data to be processed for environment detection is extremely complex. Artificial intelligence methods are, therefore, necessary to ensure a high level of traffic safety.

Fast, reconfigurable, safe and energy-efficient

For this purpose, Fraunhofer IIS and its partners in the "KI-FLEX" project are developing a powerful hardware platform and the corresponding software framework. The algorithms used for sensor signal processing and sensor data fusion are based mainly on neural networks and allow the precise detection of the vehicle position and the environment.
The significance and usability of individual sensors vary depending on the traffic situation, weather, and light conditions. The platform must be able to adapt to this. The developers, therefore, design KI-FLEX as software-programmable and reconfigurable hardware. This means that the algorithms used for sensor evaluation are changeable while driving if conditions change. In this way, the car can react flexibly to impairments or even the failure of individual sensors.
Besides, the project team will develop suitable methods and tools to ensure the functional safety of the AI algorithms used and their interaction even in the event of reconfiguration while driving. To efficiently execute all algorithms and reconfigurations, the computing resources of the hardware platform dynamically allocate according to the workload.

Sustainable, neuromorphic technology component

The planned platform is a new development in the field of neuromorphic hardware. Its functionality inspired by the human brain and is specifically designed and optimized for the efficient use of neural networks. In particular, it takes into account the fact that, on the one hand, the product cycles in the automotive industry are very long. Still, on the other hand, AI algorithms are developing rapidly. Therefore, the project aims at a hardware platform that can easily and quickly adapt to new software and hardware requirements for machine learning.

This is to be achieved by a flexible programmable deep learning accelerator with several computing cores in the form of a self-developed, application-specific chip (ASIC). According to IIS, the use of ASICs reduces costs and power consumption compared to conventional multi-purpose processors (CPUs) or graphics processors (GPUs) - at least when the quantities are very high. The development of an ASIC is initially costly. In this respect, the project could provide a strong impetus for science and the automotive industry in the field of autonomous driving.

Project consortium of research and industry partners

The German Federal Ministry of Education and Research (BMBF) is funding the joint project "KI-FLEX" within the framework of the guideline for funding research initiatives in the field of "AI-based electronic solutions for safe autonomous driving (AI element: autonomous driving)." Scheduled to run until August 2022.
Fraunhofer IIS will lead the project consortium, which includes the research and industry partners Ibeo Automotive Systems GmbH, Infineon Technologies AG, videantis GmbH, TU Munich (Chair of Robotics, Artificial Intelligence, and Real-Time Systems), Fraunhofer Institute for Open Communication Systems FOKUS, Daimler Center for Automotive IT Innovations (DCAITI, TU Berlin) and FAU Erlangen-Nuremberg (Chair of Computer Science 3: Computer Architecture).

This article was first published in German by