2018-00632 - Computational Video Editing for Stage Performances

Type de contrat : CDD de la fonction publique

Niveau de diplôme exigé : Bac + 5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

Le centre de recherche Inria Grenoble Rhône-Alpes regroupe un peu moins de 800 personnes réparties au sein de 35 équipes de recherche et 9 services support à la recherche.

Ses effectifs sont distribués sur 5 campus à Grenoble et à Lyon, en lien étroit avec les laboratoires et les établissements de recherche et d'enseignement supérieur de Grenoble et Lyon, mais aussi avec les acteurs économiques de ces territoires.

Présent dans les domaines du logiciel, du calcul haute performance, de l'internet des objets, de l'image et des données, mais aussi de la simulation en océanographie et en biologie, il participe au meilleur niveau à la vie scientifique internationale par les résultats obtenus et les collaborations tant en Europe que dans le reste du monde.

Contexte et atouts du poste

Context : This PhD thesis is proposed as part of an ongoing collaboration between computer scientists and performings arts researchers at Univ. Grenoble Alpes and INRIA to use video in teaching and researching the performing arts. In a previous project, the IMAGINE team at LJK and INRIA developped methods for automatic generation of cinematic rushes from ultra high definition video recordings of stage performances [1].

Here, we would like to propose techniques for making documentary movies from the generated rushes, based on an analysis of the script of the performance and a formalization of the rules of film editing. Ideally, the proposed techniques should be completely non-invasive (not requiring sensors on actors or on stage) and intuitive enough to be used by performing arts students, professors and researchers, without any expertise in video production.

References:

[1] Vineet Gandhi, Rémi Ronfard, Michael Gleicher. Multi-Clip Video Editing from a Single Viewpoint. CVMP 2014 - European Conference on Visual Media Production, Nov 2014.

Mission confiée

Description: The goal of the PhD thesis will be to propose novel interaction techniques to students, professors and researchers in the performing arts for making movies from stage performances recorded on stage. On the one hand, we will propose novel algorithms for editing cinematographic rushes together into movie clips automatically, based on computational models of film editing « idioms » and machine analysis of the actors speech and motion. On the other hand, we will propose novel user interfaces for easily choosing between available idioms as in [2] and creating new idioms for the specific purpose of teaching and researching mise en scene and acting techniques.

During his/her thesis, the PhD student will create an extensive database of stage performance recordings, as part of a collaboration with the performing arts department at Univ. Grenoble Alpes and associated theatre companies. The raw recordings and the generated movies will be used as supporting material for teaching mise- en-scène and acting techniques, and for researching multiple aspects of expressive human motion, verbal and non-verbal communication, and dramaturgic techniques, as part of the new crossdisciplinary research project « Performance Lab ».

References:

[2] Mackenzie Leake, Abe Davis, Anh Truong, and Maneesh Agrawala. Computational video editing for dialogue-driven scenes. ACM Trans. Graph. 36, 4, July 2017.

 

Principales activités

Research, design and implement algorithms for video editing using recordings of theatre performances annotated with (incomplete) actors identities and poses. This will include implementing state-of-the-art speech recognition and speaker diarization methods for recognizing speaking actions and adapting them to the case of live performances.

Design and implement graphical user interfaces for interacting with the above algorithms.

Create and annotate datasets of rushes and edited movies suitable for training and testing the above methods.

Evaluate results both on an objective and a subjective basis.

 

Compétences

Languages : Candidate must be fluent in French and English.

Open-minded : Candidate must have in interest in performing arts (theatre, dance and music) and be willing to work in a highly inter-disciplinary group of researchers in computer science, arts and humanities.

 

Avantages sociaux

  • Restauration subventionnée
  • Transports publics remboursés partiellement
  • Sécurité sociale
  • Congés payés
  • Aménagement du temps de travail
  • Installations sportives