PhD Position F/M PhD Thesis on RL-based Decision-Making and Planning for Automated Driving
Type de contrat : CDD
Niveau de diplôme exigé : Bac + 5 ou équivalent
Fonction : Doctorant
Contexte et atouts du poste
You will work within the ASTRA team (Automated and Safe TRAnsportation systems), a joint team of scientists from Inria and Valeo led by Fawzi Nashashibi (Inria) and Benazouz Bradai (Valeo).
This team designs models and algorithms for the development of architectures for intelligent transport systems. It is involved in several projects financed by the French National Research Agency, which aim to welcome Valeo employees and recruit young PhD students.
Mission confiée
During the PhD thesis, the general objective is to:
• Develop an RL-based decision-making and planning framework for automated driving systems. It is only natural to begin turning to the standard model for sequential decision making: the Markov Decision Process. At first glance, this framework shines with its simplicity and elegance, but also its apparent generality and representation power. An (observable) state space, a (hierarchical) action space, a (quasi-linear) system dynamics and a (dense) reward function are to be addressed for a large class of behavioural planning tasks.
• Optimize driving policies using RL algorithms to ensure safety, efficiency, and adaptability. The study in [8] explores several fundamental algorithms in Deep RL to improve automated driving performance, namely Proximal Policy Optimization (PPO), Deep Q-network (DQN), and Deep Deterministic Policy Gradient (DDPG). The paper documents a comparative analysis of these three prominent algorithms—based on their speed, accuracy, and overall performance. After a thorough evaluation, the research indicates that the DQN surpassed the other existing algorithms
• Evaluate the performance of the proposed RL system through simulations and real-world testing.
This is achieved regarding the following key points:
– Create a diverse set of driving scenarios representative of real-world conditions, including highway driving, urban environments, intersections, pedestrian crossings, adverse weather conditions, etc.
– Integrate the RL algorithm into a simulation/testing environment including real vehicle dynamics, sensor inputs, environmental factors, and interactions with other agents (e.g., vehicles, pedestrians).
– Define relevant performance metrics to evaluate the RL system’s behavior. Metrics may include safety (e.g., collision rate), efficiency (e.g., average speed, fuel consumption), adherence to traffic rules, and comfort (e.g., smoothness of maneuvers).
– Compare the performance of the RL-based system against baseline methods, such as rule-based controllers or handcrafted algorithms, to demonstrate its superiority.
In conclusion, the proposed research project holds immense potential in advancing RL applications in automated driving systems by developing a sophisticated decision-making framework that prioritizes safety, efficiency, and adaptability. Through rigorous evaluation and testing, this project aims to contribute valuable insights to the field of autonomous vehicle technology, ultimately leading to safer, more efficient, and intelligent autonomous driving solutions.
References
[1] Laurène Claussmann, Marc Revilloud, Dominique Gruyer, and Sébastien Glaser. A review of motion planning for highway autonomous driving. IEEE Transactions on Intelligent Transportation Systems,21(5):1826–1848, 2020.
[2] Fernando Garrido and Paulo Resende. Review of decision-making and planning approaches in automated driving. IEEE Access,10:100348–100366, 2022.
[3] D. González, J. Pérez, V. Milanés, and F. Nashashibi. A review of motion planning techniques for automated vehicles. IEEE Transactions on Intelligent Transportation Systems, 17(4):1135–1145, April 2016.
[4] B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(6):4909–4926, 2022.
[5] Hanna Krasowski, Yinqiang Zhang, and Matthias Althoff. Safe reinforcement learning for urban driving using invariably safe braking sets. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 2407–2414, 2022.
[6] Shahrokh Paravarzar and Belqes Mohammad. Motion prediction on self-driving cars: A review, 2020.
[7] Stefano Pini, Christian S. Perone, Aayush Ahuja, Ana Sofia Rufino Ferreira, Moritz Niendorf, and Sergey Zagoruyko. Safe real-world autonomous driving by learning to predict and plan with a mixture
of experts. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10069–10075, 2023.
[8] Akshaj Tammewar, Nikita Chaudhari, Bunny Saini, Divya Venkatesh, Ganpathiraju Dharahas, Deepali Vora, Shruti Patil, Ketan Kotecha, and Sultan Alfarhood. Improving the performance of autonomous driving through deep reinforcement learning. Sustainability, 15(18), 2023.
[9] Shijie Wang and Shangbo Wang. A novel multi-agent deep rl approach for traffic signal control, 06 2023
Principales activités
Main activities:
- research
- write scientific papers
- present work at scientific conferences
- programming
- data management and curation
- interact with partners (scientists, engineers)
- write a doctoral thesis
- participate to demonstrations and showcases
Compétences
Candidate Profile :
- Master’s Degree in the relevant field
- Strong background in machine learning, particularly reinforcement learning
- Proficiency in programming languages such as Python, C++
- Solid understanding of robotic systems and control challenges real-time performance and for real-world environments
- Familiarity with automated vehicles simulation environments
- Experience in developing control algorithms or motion planning strategies for autonomous vehicles
- Excellent problem-solving skills and the ability to work both independently and collaboratively in an interdisciplinary team.
Languages :
- French optional but very desirable.
- Good level of English for communication (international team).
Additional skills:
- Strong ability to work in groups,
- autonomy [essential],
- motivation,
- strength of initiative
Avantages
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours (after 6 months of employment)
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Informations générales
- Thème/Domaine :
Robotique et environnements intelligents
Ingénierie logicielle (BAP E) - Ville : Paris
- Centre Inria : Centre Inria de Paris
- Date de prise de fonction souhaitée : 2024-10-01
- Durée de contrat : 3 ans
- Date limite pour postuler : 2024-07-17
Attention: Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.
Consignes pour postuler
Sécurité défense :
Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement.
Politique de recrutement :
Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.
Contacts
- Équipe Inria : ASTRA
-
Directeur de thèse :
Nashashibi Fawzi / Fawzi.Nashashibi@inria.fr
A propos d'Inria
Inria est l’institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l’interface d’autres disciplines. L’institut fait appel à de nombreux talents dans plus d’une quarantaine de métiers différents. 900 personnels d’appui à la recherche et à l’innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'efforce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.