PhD Position F/M Learning to Translate Freehand Design Drawings into Parametric CAD Programs
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
About the research centre or Inria department
The Inria centre at Université Côte d'Azur includes 37 research teams and 8 support services. The centre's staff (about 500 people) is made up of scientists of different nationalities, engineers, technicians and administrative staff. The teams are mainly located on the university campuses of Sophia Antipolis and Nice as well as Montpellier, in close collaboration with research and higher education laboratories and establishments (Université Côte d'Azur, CNRS, INRAE, INSERM ...), but also with the regiona economic players.
With a presence in the fields of computational neuroscience and biology, data science and modeling, software engineering and certification, as well as collaborative robotics, the Inria Centre at Université Côte d'Azur is a major player in terms of scientific excellence through its results and collaborations at both European and international levels.
Context
This Ph.D. is funded by a joint ANR-NSF grant, in collaboration with Daniel Ritchie from Brown University
Context and Objective
Computer Aided Design (CAD) is a multi-billion dollar industry responsible for the digital design of almost all manufactured goods. It leverages parametric modeling, which allows dimensions of a design to be changed, facilitating physically-based optimization and design re-mixing by non-experts. But CAD’s potential is diminished by the difficulty of creating parametric models: in addition to mastering design principles, professionals must learn complex CAD software interfaces.
To promote effective modeling strategies and creative flow, design educators advocate freehand drawing as a preliminary step to parametric modeling. Unfortunately, CAD systems do not understand these drawings, so designers must re-create their entire design using complex CAD software. Can we automatically convert freehand drawings to parametric CAD models? Sketch-based modeling techniques do not produce parametric CAD programs [1]; classic CAD reverse-engineering techniques cannot handle drawings as input [2]; the newer field of visual program induction is promising but has been demonstrated only on simple shapes and programs [3]. By leveraging the visual vocabulary shared by drawing and CAD modeling, we will develop a system to translate from the natural language of drawing to the formal language of CAD.
References
[1] Symmetry-driven 3D Reconstruction from Concept Sketches. Felix Hähnlein, Yulia Gryaditskaya, Alla Sheffer, Adrien Bousseau. SIGGRAPH 2022.
[2] InverseCSG: automatic conversion of 3D models to CSG trees. T. Du, J. Inala, Y. Pu, A. Spielberg, A. Schulz, D. Rus, A. Solar-Lezama, W. Matusik. ACM Trans. on Graphics (SIGGRAPH Asia) 2018.
[3] Neurosymbolic Models for Computer Graphics. Daniel Ritchie, Paul Guerrero, R. Kenny Jones, Niloy Mitra, Adriana Schulz, Karl D. D. Willis, and Jiajun Wu. Eurographics 2023 STAR.
Assignment
To handle drawings as input, we will treat them as timestamped sequences of strokes, allowing us to cast the problem as one of machine translation from drawing stroke sequences to CAD program token sequences. We observe that drawing strokes are grouped into coherent drawing operations that are correlated with CAD modeling strategies (e.g. first drawing construction lines and simple primitives shapes, then refining). We propose to extract these drawing operations as an intermediate representation, which helps disambiguate between the (potentially infinitely) many programs which can represent a single shape. Performing this extraction and then producing CAD programs are complex search problems; we will leverage novel deep neural networks to guide the search.
Main activities
We will first create a dataset to study how designers draw parametric shapes. To do this, we will collect existing parametric shapes from public datasets and hire professional designers to draw these shapes. These drawings will be used to test our algorithms.
We will then develop a drawing analysis algorithm in order to identify the lines of a drawing which correspond to the same 3D parametric operation, then recognize this operation and deduce its parameters. To do this, we intend to use machine learning algorithms that we will train with synthetic drawings generated from reference parametric 3D shapes.
Finally, we will develop a program synthesis method which will combine all the estimated information (input design, recognized 3D operations, estimated parameters) and which will generate a valid program best reproducing the design while being modifiable in the existing 3D design tools. We plan to use a Transformer learning architecture, originally developed for text translation, to “translate” the drawing sequence into a sequence of 3D instructions.
Skills
The candidate must have experience in Python programming. Past experience in implementing 3D modeling or 3D analysis algorithms is a plus.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Remuneration
Duration: 36 months
Location: Sophia Antipolis, France
Gross Salary per month: 2100€ brut per month (year 1 & 2) and 2190€ brut per month (year 3)
General Information
- Theme/Domain :
Interaction and visualization
Software engineering (BAP E) - Town/city : Sophia Antipolis
- Inria Center : Centre Inria d'Université Côte d'Azur
- Starting date : 2024-10-01
- Duration of contract : 3 years
- Deadline to apply : 2024-07-05
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Contacts
- Inria Team : GRAPHDECO
-
PhD Supervisor :
Bousseau Adrien / Adrien.Bousseau@inria.fr
The keys to success
The successful candidate should have knowledge in computer graphics and/or computer vision and be interested in 3D modeling and related topics (geometry processing, numerical optimization, machine learning).
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.