Virtual Inspection Planning – Paradigm Shift Through Simulation

Factories are becoming increasingly automated. Production facilities are becoming more flexible, so that no new plants have to be built when switching to new products. Visual inspection system hardware configuration – the last nail preventing the inspection systems to be truly purpose-flexible and thus ready for the implementation as a part of Industry 4.0 process chain. Production lines are more and more versatile, and products are changing rapidly, confronting inspection systems with more complex surfaces and materials. Every step of the production is controlled and digitalized to be as flexible as possible. And yet, when it comes to inspection, months of pre study are required, and no off-the shelf solution is available which can be easily adapted to different use cases and surfaces of different complexity.


Making Processes more Flexible and Efficient

Quality control is often neglected when talking about Industry 4.0. Inspection systems are still rigid and have to be designed for specific products. An inspection system consists of many hardware components, typically selected and parameterized by experienced engineers on the basis of physical tests. New systems are developed iteratively. Experts design an initial system, which is then modified until it can inspect the product with sufficient accuracy. These tests of different hardware solutions cost a lot of time and effort - several hours per test run. Therefore, a configuration is often chosen that works but is not optimal. The resulting sub-optimal image quality must be algorithmically compensated later.

To make this process more flexible and efficient, we are developing an adaptive, simulation-based framework that will revolutionize the development process for inspection systems. In the future, industrial inspection systems will be completely virtual designed and tested for reliability using this framework.

The working group
© Fraunhofer ITWM
The working group consists of Prof. Hans Hagen, Josiah Abah, Dr. Petra Gospodnetic, Duje Štolfa, Juraj Fulir and Lovro Bosnar. Markus Rauhut is missing on the photo.

Inspection Planning Framework

The Virtual Inspection Planning Research Group of our institute is set on changing the paradigm by developing a modular framework fully capable of planning the acquisition requirements to completely inspect any given product. Using computer vision, computer graphics, machine learning and robotics it is possible to develop a framework offering tools for design optimization, allowing the assumption of a flexible image acquisition setup. Currently, very little or no research is focused on inspection system design and optimization.

Airplane Wing Detection
© Fraunhofer ITWM
Parametric texture synthesis over multiple-scales for the purpose of physically correct inspection simulation and dataset generation.

Virtual Inspection Planning framework makes it possible to overcome this gap by thoroughly testing the acquisition hardware of choice and simulating the result. Most importantly, it makes optimization of component positioning possible, without requiring the engineer to remount the equipment repeatedly. Furthermore, computer vision algorithms can be developed and tested on simulated images, along with the acquired ones, overcoming a frequent problem of defect sample acquisition. Such problems are often found in industries where defects occur rarely but are critical when they do – airplane blisks (Blade Integrated Disk) and car brakes are two examples. Therefore, we introduced a modular pipeline focusing on

  • Viewpoint generation/optimization
  • Interactive evaluation and planning
  • Defect modelling
  • Plan simulation and dataset generation
  • Physical verification

Video: The Future of Automated Vision Inspection

The research group combines the fields of image processing, computer graphics, machine learning and robotics to make optical inspection of complex products possible. The basic idea is to simulate the complete process of an optical inspection in the computer.

Virtual Inspection Planning

Video: Planning Visual Inspection for International Space Station

It has been over 20 years that ISS is orbiting the Earth. As with any other product, it's components may fail or get damaged. However, the problem is that it is hardly "at hand" to be inspected and repaired. Using the publicly available data, we have applied Inspection Planning Framework to find out what would be necessary to inspect it.