2022-05288 - Post-Doctoral Research Visit F/M Interpretable Deep Learning for image-based species identification

Contract type : Fixed-term contract

Level of qualifications required : PhD or equivalent

Fonction : Post-Doctoral Research Visit

About the research centre or Inria department

The Inria Sophia Antipolis - Méditerranée center counts 34 research teams as well as 7 support departments. The center's staff (about 500 people including 320 Inria employees) is made up of scientists of different nationalities (250 foreigners of 50 nationalities), engineers, technicians and administrative staff. 1/3 of the staff are civil servants, the others are contractual agents. The majority of the center’s research teams are located in Sophia Antipolis and Nice in the Alpes-Maritimes. Four teams are based in Montpellier and two teams are hosted in Bologna in Italy and Athens. The Center is a founding member of Université Côte d'Azur and partner of the I-site MUSE supported by the University of Montpellier.

Context

Methodological context: XAI

With data-driven AI methods becoming prevalent in more and more aspects of our life, there is a growing effort within the AI community to ensure that these methods do not only perform well, but also communicate to humans some elements of the learned internal reasoning that leads to a particular output. This effort towards Explainable AI (XAI) [1] aims at ensuring that humans can understand the process within the AI model, ultimately allowing for a more human-centric AI. The vast majority of methods, those based on feature attribution, aim at pointing out which parts of the input are most responsible for the output, potentially resulting in an ambiguous interpretation that may end up being misleading [2, 3]. We want to explore how to achieve richer and more useful types of explanations, such as those using prototypical examples [4] or natural language [5], and explore the impact that these explanations have on the users in terms of trust and of how much the users are able to learn from the system. In addition, rich explanations, such as those in the form of visual characteristics in natural language, can be designed to provide an interpretable representation of each data sample that can be leveraged for few-shot and zero-shot learning, as long as the characteristics of new classes are known.

Application context: Species ID and biodiversity mapping

The methods proposed within this project  will be devoted to the analysis of real world data for biodiversity ecosystems assessment and monitoring at different spatial scales. The first task is related to the automatic identification of plant species in user-generated photographs with the goal to pave the way towards better Species Distribution Models [6] and even enable us to monitor fine-grained trends in species distribution shifts caused by climate change or other anthropogenic pressures. The computer vision models for species identification used by platforms such as iNaturalist [7] and Pl@ntNet [8] have improved enormously in the last few years and can often be trusted when presented with high quality photographs of data-rich species. However, they struggle with images taken under suboptimal conditions (bad lighting, blurriness, partial news, etc.) or of species for which not many images are available. In such cases an incorrect automatic label may contribute more to adding noise to the platform than no label at all, since new users tend to overtrust [9] the computer vision model. New computer vision methods are needed that perform better on identifying uncommon species and that allow users to understand when they should be trusted and when not. We will develop new methods that leverage species descriptions based on morphological traits in order to achieve these goals.The second task concerns the analysis of remote sensing imagery to predict habitat (or landscape) characteristics. In a similar fashion to how morphological traits can be used to describe species, these landscape characteristics can be used to describe specific habitats. We will thus explore the development of description-based XAI methods that are useful at both scales.

Assignment

In order to improve the performance on data-poor species and to help users gauge which automatic predictions to trust, we aim at designing new computer vision methods that work more like an expert taxonomist or ecologist: by first looking for multiple morphological traits on the image and then finding which species are compatible with the observed traits in the case of species identification, or by looking for habitat characteristics in the case of habitat characterization. To fulfill this, the project will pursue two different methodological goals: (1) explore the use of natural language bottlenecks describing visible traits or other visual characteristics and (2) design computer vision models that discover discriminative visual characteristics in an unsupervised manner.

Main activities

Within the first goal, the first task will consist of exploiting a recently created dataset of morphological traits per species for several tens of thousands of species, obtained automatically by parsing textual species descriptions from the web. The noisy nature of this dataset will require the development of noise robust vision/text models that learn to detect traits on images. This will be followed by the development of an additional model that is able to infer species probabilities by looking at the detected traits.

The second goal will help us discover visual traits that, although important for the identification, have not been included in the original trait dataset. To do this, we will work towards the development  of computer vision architectures that are able to discover trait-like visual characteristics, both in a weakly supervised setting, using only images and their species labels, and by leveraging the existing trait annotations.

Skills

Technical skills and level required :

  • Good command of Python programming.
  • Understanding of Deep Learning methods (incl. good basis of algebra, calculus, etc.)
  • Experience with Pytorch, Tensorflow or similar.

Languages : Fluency in spoken and written English.

Relational skills : We are seeking a self-motivated researcher, able to work autonomously while collaborating with the rest of the team and co-supervising one or more PhD students.

Other values appreciated : We value passion for both method development and the advancement of biodiversity knowledge.

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage
  • Supplementary social protection

Remuneration

Gross Salary: 2746 € per month