PhD Position F/M Deformable object manipulation by "photometric shape servoing"

Contract type : Fixed-term contract

Level of qualifications required : Graduate degree or equivalent

Fonction : PhD Position

About the research centre or Inria department

The Inria Centre at Rennes University is one of Inria's eight centres and has more than thirty research teams. The Inria Centre is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.



  • The Ph.D. position is full-time for 3 years (standard duration in France). The position will be paid according to the French salary regulations for PhD students.

Supervisors: Alexandre Krupa, Eric Marchand



A major challenge in robotics is the interaction with deformable objects. Indeed, nowadays most of the robot control frameworks are built for rigid objects observed in the scene. Therefore, in order to extend the manipulation capabilities of robots, the main objective of this thesis is to investigate a new robotic control approach for manipulating soft objects to apply a desired deformation. The general idea is to use visual feedback to estimate and track in real time the deformations of the object of interest, and to develop a visual servo-control approach to control one or more robotic manipulators in order to apply a desired deformation to the object.

Automatically controlling the deformation of soft object would enable numerous robotic applications such as gripping a deformable object with a robotic hand, assembling flexible elements, accurately manipulating elastic materials or food products.

Recently, a number of studies have proposed solutions for applying controlled deformations to soft objects by robots, leading to the emergence of a new field of robotics research that is called "Shape servoing".

Controlling the deformation of soft materials requires the knowledge of how displacements of the robotic manipulator are translated into material deformation. This relationship can be either estimated from past visual observation with data-driven approaches [1] or expressed by a physics-model of the object such as finite element [2] or mass-spring models [3-4].


The various existing approaches need to extract visual geometric features to represent the object current deformations. For example they use 3D points of the objects surface provided by an RGB-D camera. Therefore, one of their limitations is the requirement of a real-time process that extracts and tracks these geometric features. This process usually involves a segmentation process which is only optimized and efficient for a specific type of object.

To avoid the need for extracting geometric features, we will propose in this thesis to consider directly the photometric information as well as the depth information provided by the RGB-D camera as the visual features to be regulated by the control law. This was already done for positioning a camera with respect to a rigid object by the “direct visual servoing” concept introduced in [5-6]. However, at our best knowledge photometric features were never considered for shape visual servoing.


[1] R. Lagneau, A. Krupa, M. Marchal. Automatic Shape Control of Deformable Wires based on Model-Free Visual Servoing. IEEE Robotics and Automation, 5(4):5252-5259, October 2020.
[2] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit and C. Duriez, « FEM-Based Deformation Control for Dexterous Manipulation of 3D Soft Objects, » In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’18, Pages 4007-4013, Madrid, Spain, October 2018.
[3] F. Makiyeh, F. Chaumette, M. Marchal, A. Krupa. Shape Servoing of a Soft Object Using Fourier Series and a Physics-based Model. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’23, Pages 6356-6363, Detroit, USA, October 2023.
[4] F. Makiyeh, M. Marchal, F. Chaumette, A. Krupa. Indirect Positioning of a 3D Point on a Soft Object Using RGB-D Visual Servoing and a Mass-Spring Model. In Int. Conf. on Control, Automation, Robotics and Vision, ICARCV’22, Singapore, December 2022.
[5] C. Collewet, E. Marchand, F. Chaumette. Visual servoing set free from image processing. In IEEE Int. Conf. on Robotics and Automation, ICRA’08, Pages 81-86, Pasadena, Californie, Mai 2008.
[6] C. Collewet, E. Marchand. Photometric visual servoing. IEEE Trans. on Robotics, 27(4):828-834, August 2011.
[7] E. Marchand. Direct visual servoing in the frequency domain. IEEE Robotics and Automation Letters, 5(2):620-627, April 2020.
[8] E. Marchand. Subspace-based Visual Servoing. IEEE Robotics and Automation Letters, 4(3):2699-2706, July 2019.

Main activities

Envisaged activities:

This thesis will therefore focus on the elaboration of a new shape servoing framework that we call “photometric shape servoing”. One of the scientific contributions will concern the modelling of the interaction that provides the variation of the observed photometric information with respect to the motion of the robot acting on the soft object. This interaction model will then be considered in the design of a robot controller that will allow the autonomous shaping of a soft object by several robotic arms. A reduction of the dimensionality of the photometric shape features will be also considered thanks to projection approaches [7-8].

The methods studied will be developed, tested and validated on an experimental bench consisting of deformable objects, depth cameras, 2 robotic arms with 6 degrees of freedom, each equipped with an force sensor and a robotic griper/hand.

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Possibility of teleworking (90 days per year) and flexible organization of working hours
  • Partial payment of insurance costs


Monthly gross salary amounting to 2100 euros for the first and second years and 2190 euros for the third year.