PhD Position F/M Structures motion estimation by an alternative optical flow method combining sparse and particle approaches

The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.

Presentation of the Teams

The project is carried out by the I4S research team, which is a joint team between Inria and Gustave Eiffel University.

Inria: Inria is the French institute for research in digital science and technology. This public research center of scientific excellence is at the forefront of digital transformation in France. Research in computer science, mathematics, artificial intelligence (AI), software development, innovation in high-impact technological disciplines, and entrepreneurial risk (DeepTech) are the DNA of the institute. Inria is ranked 16th in the global "AI Research" ranking and is the first European institute for exploratory research in digital science.

Gustave Eiffel University: Gustave Eiffel University is a French, experimental, and multi-campus university specializing in the study of cities and urbanization processes. It is committed to promoting academic excellence, cutting-edge research, and innovation, while fostering interdisciplinary collaboration and partnerships with industry and civil society stakeholders. A leader in sustainable cities in France, it represents a quarter of French research on this theme. It has unique experimental platforms and new energy to address the challenges of the future city, as well as economic, environmental, energy, and societal transitions.

I4S: The I4S team (Inferences for Structures) specializes in Structural Health Monitoring (SHM), i.e. attempting to predict, categorize, locate, and quantify defects or damage that may occur in civil, electrical, (bio)-mechanical, energy, or aeronautical structures, etc. To do this, it is necessary to combine multiple data and expertise, such as numerical physical simulation, data processing, sensor development, embedded electronics integration, statistical uncertainty propagation, etc.

General Context

Following recent tragic events (Gênes, Mirepoix, ...), a Senate report (2019) highlighted the aging health status of infrastructure in France and the need to quickly assess the health of structures at any time. Currently, structure monitoring is ensured through regular visual inspections and locally through sampling and coring on the structure. This type of monitoring approach has certain limitations in terms of the number of structures inspected, responsiveness, and duration. In parallel, a non-destructive approach has been initiated for many years by instrumenting structures with various sensors. Access to the internal state of the structure is less direct and requires specific skills, but this approach is preferable and sustainable.
The instrumental approach has given rise to the discipline of Structural Health Monitoring (SHM), which aims in the long term to achieve the following 4 objectives: detection and localization of damage, identification/categorization, and quantification of damage intensity. Conventional instrumentation uses a network of point sensors, e.g. accelerometers, strategically placed to obtain the vibratory response of the structure through advanced signal processing techniques. Analysis tools then allow monitoring of the vibratory signature and thus fulfilling the 4 objectives of SHM [6].
In the long run, point sensor-based monitoring may limit the instrumental approach in terms of detection, analysis, and implementation cost. In this context, the I4S team has been interested for several years in the alternative solution of using continuous or distributed sensors. In particular, non-contact imaging systems allow a full-field analysis of the structure, e.g. TLS laser system, GB-SAR hyperfrequency radar, video sensor. Among these solutions, video sensors have gained renewed interest over the past decade, thanks to combined technological advances in terms of resolution, cost, acquisition and processing speed, and storage capacity. Using specific image processing techniques, video sensors allow cost-effective evaluation of structure displacement at any point in the image with subpixel accuracy [8].
The I4S team has been involved in this SHM research field since 2018 and has identified several numerical methods, the combination of which could contribute to improving motion estimation. These improvements would also make it possible to consider the possibility of merging them with vibrational signal analysis techniques such as Stochastic Subspace Identification (SSI) to design imaging-based SHM methods.

Thesis Objectives and Directions

The main objective of the thesis is to enable more efficient SHM analysis of civil engineering structures using video sensors in real conditions. To achieve this, the thesis proposes to develop innovative alternative treatments to improve the first step of the SHM process, namely, the selection of areas/pixels of interest in the image, and the actual motion estimation. Finally, the thesis will explore the possibility of simplifying the imaging-based SHM process by combining several steps.

1. Determination of areas of interest and/or active pixels by LRR:
In the approach used in ongoing work, motion is quantified based on areas of interest in the image, i.e. boundary elements of the structure and/or active pixels whose intensity varies the most in the sequence of video images. The selection of areas/pixels requires specific preprocessing. As an alternative to current preprocessing methods, this thesis will evaluate the family of methods called Low Rank Representation (LRR) [4, 7]. They are based on a sparse representation of the video image sequence and allow for a drastic reduction in the dimension of the data space. These methods have led to significant research activity [1] and find applications in many areas involving computer vision and also have a major impact on big data. The I4S laboratory conducted a preliminary study on this subject in 2022 and tested one of the many variants of this family, i.e. Go Decomposition (GoDec), on low-resolution images.

2. Structure motion estimation by LBM:
As an alternative to the currently used optical flow method [5], the thesis will seek to evaluate the use of the Lattice Boltzmann Method (LBM) to estimate the motion of the structure. This particle method offers a formalism generalizing the optical flow method and, in comparison, has the advantage of taking into account the image before and after the current image in the algorithm. In addition, LBM does not need to use markers in the image to estimate motion [2, 3]. In the thesis framework, LBM could provide higher measurement precision than conventional optical flow, without additional computational overhead. Motion calculation would be limited to the sparse pixels/zones identified by the LRR preprocessing introduced earlier. A statistical analysis will determine the motion estimation uncertainties with the developed methods, as well as the minimum vibration amplitude of the structure necessary to perform subsequent SHM analyses.

3. Fusion of the first 2 steps of the SHM process:
The steps of estimating the motion of the structure and its vibrational analysis by SSI are currently performed sequentially. Building on image vectorization, this thesis will explore the possibility of performing SSI directly from the vectorized series of video images. The large size of this 2D matrix may pose a significant challenge to the feasibility of this approach.

The creation of a test cohort will allow a comprehensive evaluation of different algorithms systematically in various conditions (synthetic images with and without noise, with oscillating or random movements, laboratory cases, and in-situ image sequences). This cohort will leverage the various experiments already conducted in the laboratory and will be enriched throughout the thesis. Special attention will be paid to the noise sensitivity of this new algorithm and the calculation speed.

References

[1] Thierry Bouwmans, Sajid Javed, Hongyang Zhang, Zhouchen Lin, and Ricardo Otazo. On the Applications of Robust PCA in Image and Video Processing. Proceedings of the IEEE, 106(8):1427–1457, August 2018.
[2] Guang-Tai Ding, Shu-Qing Li, and Dan-Xia Luo. Optical Flow Analysis Based on Lattice Boltzmann Method and Lower Order Approximation with Relaxation Factors. In 2010 International Conference on Multimedia Technology, pages 1–4, October 2010.
[3] Guang-Tai Ding, Dan-Xia Luo, and Shu-Qing Li. Image sequence segmentation based on the formal lattice Boltzmann equation and its lower order approximation. In 2010 International Conference on Audio, Language and Image Processing, pages 815–820, Shanghai, China, November 2010. IEEE.
[4] Zhi Gao, Ruifang Zhai, Pengfei Wang, Xu Yan, Hailong Qin, Yazhe Tang, and Bharath Ramesh. Synergizing Appearance and Motion With Low Rank Representation for Vehicle Counting and Traffic Flow Analysis. IEEE Transactions on Intelligent Transportation Systems, 19(8):2675–2685, August 2018.
[5] Jaka Javh, Janko Slavič, and Miha Boltežar. The subpixel resolution of optical-flow-based modal analysis. Mechanical Systems and Signal Processing, 88:89–99, May 2017.
[6] Alexander Mendler, Michael Döhler, Carlos E Ventura, and Laurent Mevel. Localizability of damage with statistical tests and sensitivity-based parameter clusters. Mechanical Systems and Signal Processing, 204:110783, 2023.
[7] John Wright and Yi Ma. High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications. Cambridge University Press, 1 edition, January 2022.
[8] Bian Xiong. Video-Based Vibration Analysis for Structural Health Monitoring in Civil Engineering. PhD thesis, Rennes 1, Nantes, France, December 2021.

Technical skills and level required : A certain autonomy in programming languages (Matlab, Python, C++ and/or Julia) is required.  a synthetic mindset and writing skills are expected.

Languages : English (French would be an asset)

Other valued appreciated : Experience in motion detection would be a plus.

• Subsidized meals
• Partial reimbursement of public transport costs
• Possibility of teleworking (90 days per year) and flexible organization of working hours
• Partial payment of insurance costs

Monthly gross salary amounting to 2100 euros for the first and second years and 2200 euros for the third year