In the DIAS project, ASAP equipped its own vehicle fleet with intelligent sensors and set up the necessary IT infrastructure.
In the DIAS project, ASAP equipped its own vehicle fleet with intelligent sensors and set up the necessary IT infrastructure.
( Source: gemeinfrei / Unsplash)

AI, Big Data & Cloud Digital Automotive Services - Pioneering Smart Cities

Author / Editor: Sebastian Heinemann and Martin Kreyling * / Florian Richert

Driverless cars that navigate independently, are intelligent and networked - thanks to artificial intelligence methods, Big Data and Cloud Computing technologies in the development of new mobility services, this goal is a big step closer.

Arrived at your destination by car - and every parking space is occupied. Almost everyone is familiar with this everyday situation, and the search for a solution is an omnipresent topic due to the continuously growing volume of traffic in urban areas. OEMs and suppliers have been working for years to ensure, for example, that drivers are informed in good time about available parking spaces near their destination. As a development partner of the automotive industry, the ASAP Group is also addressing this issue and initiated the internal development project "Digital Automotive Services" (DIAS) a few years ago. The goal: retrofittable systems, combined with intelligent services - to save the driver the search for a parking space, among other things.

In the DIAS project, ASAP is concerned with the continuous exchange of data between vehicles and an in-house back-end as well as the modification and use of the acquired black data to generate new services for the driver. For the project, the company equipped its vehicle fleet with intelligent sensors. Through the use of artificial intelligence methods, big data and cloud computing technologies, the acquired black data provide new insights and pave the way for future mobility solutions for the realization of smart cities.

Continually growing traffic volumes pose enormous challenges for cities and put drivers' patience to the test. Solutions for this are seen, among other things, in the minimization of after-hours traffic or the search for a parking space - for example, through new mobility services such as a swarm-based parking space search. They are therefore an important component of smart city concepts aimed at improving the quality of life of their residents. The creation of a suitable infrastructure - above all a suitable IT infrastructure in and around vehicles - plays a central role in this. To turn smart city concepts into reality, developers in the DIAS project are working on new approaches for future mobility services.

Swarm data enable new mobility services

DIAS aims to be able to offer digital services and functions around the vehicle in a customer-specific way. The development project provides a basis for decision-making for algorithms and IT infrastructures in the field of connected cars. The development project deals with various research areas: on the one hand, this includes the analysis of possible applications of technologies such as Big Data or Cloud Computing. The acquisition of new insights when combining data from different vehicles, for example deriving a realistic driver model from speed profiles of different drivers, is another area. Besides, work is being done on the development of location-based services by enriching map data with vehicle and environment data.

The DIAS project also deals with the use of artificial intelligence methods to enable the vehicle to recognize objects such as signs, other vehicles or pedestrians as well as complex traffic situations. This allows new Points of Interest (POI) to be automatically mapped and displayed in suitable applications such as fleet management. This also involves investigating which confidence can be assigned to the detected POIs, i.e. how trustworthy the information obtained is: for example, how often a construction site must have been detected to trigger automatic mapping or removal of the POI.

To implement the project, ASAP equipped its vehicle fleets in Ingolstadt and Munich with intelligent sensors and set up the necessary IT infrastructure. Thus, data from the vehicles are sent to a back-end and then made available to the users of the services again in prepared form. With the black data obtained in this way, knowledge about meaningful application possibilities of algorithms in the field of machine learning and object recognition is gained. Also, Big Data technologies can be evaluated and analyzed about possible applications for customer projects.

Figure 1: Structure, architecture and interrelationships in the DIAS project
Figure 1: Structure, architecture and interrelationships in the DIAS project
(Source: ASAP)

Contexts and Architecture in the DIAS Project

In the DIAS development project, the infrastructure for the collection, transmission, administration, and evaluation of data was first developed and implemented. This forms the basis for the evaluation of possible applications of the acquired data and for a rapid prototyping environment to implement new services in an uncomplicated and practical way. For this purpose, technologies and current frameworks from the areas of Big Data and Cloud Computing were implemented. Figure 1 illustrates the architecture and interrelationships within the DIAS project:

  • Devices: Capturing data from the vehicle and the environment, such as current vehicle speed, driver interactions, and speed limits,
  • Communication: Encrypted, continuous communication between devices and the cloud,
  • Cloud: IT infrastructure set up for the project,
  • Intelligence: Evaluation of the collected data with different approaches, for example, image processing, mathematical models or methods from the field of machine learning.
  • Automotive Services: New mobility services with added value for the driver.

The devices used in the vehicles were initially developed based on a Raspberry Pi. Different sensors such as cameras or GPS receivers were connected to the device. Also, hardware was developed that can be used to read out vehicle bus systems. ROS (Robot Operating System) is used as middleware. This enables standardized administration and communication of the individual software functions on the device. The distribution of software updates and configurations is realized via Puppet. This allows various measurement campaigns to be controlled centrally.

Communication with the cloud is encrypted and secured via certificate management. While the recording of vehicle and GPS data is continuous, camera data is only recorded in the software via special triggers and transferred to the backend. These triggers are triggered by object recognition running in the device, such as traffic sign recognition. To save LTE data volume, a burst mode was implemented as a ROS node. This means that certain data, such as images or environmental information, can only be transmitted at one of the ASAP locations when a WLAN connection is established. Certain vehicle signals are continuously transmitted from the vehicles to the cloud. This enables the developers to conduct measurement campaigns with live data and those with more static information adapted to the respective application.

Figure 2: Enhancement of map data with acquired vehicle and environment data
Figure 2: Enhancement of map data with acquired vehicle and environment data
(Source: ASAP)

In the cloud, the data is also received and processed via ROS nodes. The data is then stored in NoSQL databases. Each element of the cloud runs in a docker container, making it easy to perform load balancing and ensure future scalability. Received image data is forwarded to the intelligence for analysis using a GPU cluster. Recognized objects are verified with existing data, then merged and written to the database. This allows confidence to be created for each object, indicating how secure the object exists. This is particularly necessary to be able to react to changes in the environment (road construction/management, infrastructure).

The database offers a uniform interface for flexible data evaluation and preparation for the services. For example, all recognized objects can be visualized as static and dynamic POIs on a map. The services can also filter the data according to their confidence and only use those that have a sufficiently high confidence for the service.

In addition to the evaluation of data, another focus of the DIAS project's intelligence is on improving existing models or building new ones. For this purpose, the data gained is processed by the developers and used for training, evaluation, and testing. Further services can then be developed from new models. Besides, methods from the field of deep learning can be used to find connections between different data sources and then examine the connections more closely.

Combination of different methods

By using different methods - classical algorithms from image processing as well as machine learning methods - the developers are able, for example, to recognize light signal systems based on the acquired black data. ASAP feeds neural networks with data from the vehicles and trains them for specific situations. Then the learning process of artificial intelligence is validated: The new network is sent to the vehicles - where the quality of the trained algorithm is tested by determining the recognition rates of the system for the newly trained and similar situations in road traffic.

Figure 3: Graphical representation of the data recorded in the DIAS project
Figure 3: Graphical representation of the data recorded in the DIAS project
(Source: ASAP)

This will also provide insights into the form in which the training data must be processed so that the algorithms can recognize relevant situations more efficiently in the future. Light signal systems can already be automatically stored as POIs in the database system and linked with further information there. In this way, maps can be enriched with a wide variety of information and live models can be created in which the most varied details - vehicle locations, live movements including vehicle signals and locations of traffic lights - are depicted.

The DIAS development project combines conventional technologies with artificial intelligence methods to produce new findings. For example, image processing for sign recognition is combined with machine learning to identify POIs for the live models: semi-skilled algorithms recognize, for example, when many vehicles brakes at one point and change lanes - an indication of a possible construction site. In combination with sign recognition, it will be possible to identify danger zones even more clearly in the future. With the help of machine learning, realistic driver models can also be generated from the swarm data. A wide variety of information about driving behavior - when and why the driver brakes, how hard he brakes and at what speed he drives in relevant scenarios - is incorporated. Such driver models are used, for example, for test purposes in the area of virtual validation.

DIAS - Pioneer for Smart Cities and Autonomous Driving

Driverless cars that navigate independently, are intelligent and networked - new findings from the DIAS project bring ASAP a big step closer to this goal: high-quality, clear maps are one of the basic requirements for autonomous driving. The live models, in which danger points and much more information are automatically and reliably mapped, can be used for this purpose. In the future, for example, it will be possible to suggest optimal routes to drivers depending on the traffic situation before an intelligent search for a parking space leads them directly to a free parking space at their destination.

This article was first published in German by

* Sebastian Heinemann is Head of Software Development at ASAP.

* Martin Kreyling is Head of Software Development, Function and Framework Development at ASAP.