2019-01781 - Doctorant F/H Algorithmes pour la Simplification des Réseaux de Neurones

Type de contrat : CDD

Niveau de diplôme exigé : Bac + 5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Inria Sophia Antipolis - Méditerranée center counts 34 research teams as well as 8 support departments. The center's staff (about 500 people including 320 Inria employees) is made up of scientists of different nationalities (250 foreigners of 50 nationalities), engineers, technicians and administrative staff. 1/3 of the staff are civil servants, the others are contractual agents. The majority of the center’s research teams are located in Sophia Antipolis and Nice in the Alpes-Maritimes. Four teams are based in Montpellier and two teams are hosted in Bologna in Italy and Athens. The Center is a founding member of Université Côte d'Azur and partner of the I-site MUSE supported by the University of Montpellier.

Mission confiée

Assignments :
Conduct original scietific research with the goal of obtaining a Ph.D. with a thesis focusing on the project's topic.

Context :
The widespread use of neural networks on devices with computationally-low capabilities, demands for lightweight and energy-efficient networks . Despite such need, and despite the strategies employed to prevent overfitting by removing a substantial part of their edges (such as pruning and dropout [SHK14,GBC16]), the question of how to reduce their size in terms of the number of neurons appears largely unexplored [MTK16]. The aim of the project is to investigate algorithmic procedures to reduce the size of neural networks, in order to improve the speed with which they can be evaluated [CWT15] and to shed light on how much information about the computational problem at hand can be encoded within neural networks of small size [M03].
In particular, the project will focus on the design compression algorithms that, given a pre-existing network of artificial neurons, modify its structure by exploiting the high level of redundancy that is observed empirically [DSD13], and produce a network with a reduced number of neurons. We will then empirically evaluate such procedures on architectures that have proven themselves for the task at hand [R18]. An original track for this is to draw on the techniques of compact labeling schemes used to compress the encoding of distances in a graph [S17].

[MTK16] P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning Convolutional Neural Networks for Resource Efficient Inference,” arXiv:1611.06440 [cs, stat], Nov. 2016.
[CWT15] W. Chen, J. T. Wilson, S. Tyree, K. Q. Weinberger, and Y. Chen, “Compressing Neural Networks with the Hashing Trick,” in Proceedings of the 32Nd International Conference on on Machine Learning - Volume 37, Lille, France, 2015, pp. 2285–2294.
[SHK14] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014.
[GBC16] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: The MIT Press, 2016.
[M03] D. J. C. MacKay, Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
[DSD13] M. Denil, B. Shakibi, L. Dinh, M. Ranzato, and N. de Freitas, “Predicting Parameters in Deep Learning,” in Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, USA, 2013, pp. 2148–2156.
[R18] https://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html
[S17] C. Sommer. Shortest-Path Queries in Static Networks, in ACM Computing Surveys, 46(4), 2014.

Principales activités

Main activities:

  • Acquire knowledge on the topics relevant to the project: fundamentals of artificial neural networks with a focus on deep learning architectures, knowledge of the core ideas to compress the encoding of metric properties in a graph such as compact labeling schemes. 
  • Conducting original scientific research on the project's topic by designing novel algorithms, evaluating them empirically and analyzing them mathematically.

Avantages

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage

Rémunération

Duration: 36 months
Location: Sophia Antipolis, France
Gross Salary per month: 1982€brut per month (year 1 & 2) and 2085€ brut/month (year 3)