Pedestrian attention and intention recognition for safe ADAS

The main objective of AWARE2ALL is to address the new safety challenges posed by the introduction of highly autonomous vehicles in mixed road traffic through the development of inclusive and innovative safety (passive and active) and external and internal Human-Machine Interaction (HMI) systems that will consider the variety of road users and will objectively demonstrate relevant improvements in mixed traffic safety. One crucial element of this project involves the creation of an external HMI capable of dynamic vehicle interaction with the nearby pedestrians. 

To develop an effective external HMI capable of providing meaningful and contextually appropriate external feedback in alignment with the vehicle's surroundings, it is essential to take a proactive approach to the scene. This involves actively engaging with the environment to systematically gather contextual information. By doing so, feedback can be customized to adjust seamlessly with the presence, profiles, and behavioral patterns of pedestrians in the immediate surroundings.  

In developing the technology to align with the AWARE2ALL goals, a deep learning (DL) approach has been adopted. Presently, DL stands as a remarkably successful approach, particularly in addressing intricate tasks related to image processing. Furthermore, advancements in their real-time application broaden the viability of considering them as the optimal choice. Hence, two fundamental DL algorithms are currently under development, used to recognize pedestrian attention and intention, and are integral to the overarching technology being created for the project.

Attention recognition: Central to the development of the external HMI is the concept of measuring the attention levels of the pedestrians in proximity to the vehicle. This information can be instrumental in understanding the behavior of pedestrians, allowing for nuanced and timely feedback, keeping in mind the safety of the pedestrians as well as the vehicle. Utilizing deep learning models, the current version of the algorithm can extract details about the orientation of a pedestrian's body and face translating this information into their respective attention levels.  

Intention recognition: This algorithm aims to discern potential actions and intentions of pedestrians in proximity to the car. The focus lies on identifying whether pedestrians intend to cross the road, a scenario representing the most vulnerable situation on the street. Similar to attention recognition, this data is crucial for the development of a responsive and efficient HMI feedback system. By employing DL models, the algorithm can adeptly track the movements of pedestrians within the scene and extract comprehensive contextual information. The extracted information plays an essential role in accurately determining whether pedestrians have the intention to cross the road or choose not to do so.  

Application  

The primary objective of this technology is to contribute to the development of the dynamic HMI capable of adapting to the current situation, prioritizing the safety of both pedestrians and the vehicle. The algorithms previously discussed will form integral components of the HMI system, and will serve as a crucial input for determining how the vehicle will interact with its environment. For example, it will play an important role in determining the appropriate types of feedback, whether audio or visual, to be disseminated, and it will also govern the timing and recipients of such feedback. Overall, the goal behind the development of this technology is to infuse a dynamic element into the external HMI system, which is a vital necessity for effectively navigating unpredictable real-world situations enhancing the safety aspect of future ADAS. 

Do you want more information? Please contact Pritomrit Bora pritomrit.bora@ficosa.com, Cristina Pérez Benito c.perez.benito@ficosa.com, Aleksandar Jevtic aleksandar.jevtic@ficosa.com from FICOSA.