Fully automated, visual end-of-line inspection with AI performs complex controls

Intelligently tested

Fully automated, visual end-of-line inspection with AI performs complex controls

The demands manufacturers place on production equipment are enormous: They should be trouble-free and cost-efficient with the highest possible throughput, produce zero-defect quality and be highly flexible. However, conventional quality inspections often have deficits in terms of reliability, flexibility and detection rate and are relatively expensive. If quality inspection can only be carried out by the human eye, there is also a risk that it will become very tiring, exhaustive and thus imprecise. However, the more complex the testing situation is, the more challenging quality assurance is, but at the same time, the greater the potential for savings.

Application

The use of industrial cameras for visual inspection is often a first step towards automation. If artificial intelligence is added, a self-learning, self-sufficient 100% test is possible. The example of end-of-line testing of complex assemblies with multiple test characteristics shows how artificial intelligence can master even demanding situations. The Austrian company Nordfels GmbH has developed an automatic, self-learning testing machine for the visual 100% inspection of assemblies or aggregates of any kind. This "deep learning inspector" consists of an articulated arm robot, intelligent software and a GigE Vision industrial camera from IDS.

The system can be used to examine, for example, combustion engines, transmission units or fire pumps, as well as electric drive trains, EV battery systems or other components from the world of e-mobility. Everything that consists of various attachments, cables or hoses and must be checked for completeness and correctness is inspected. These are often complicated components with numerous, intricate individual parts. Manually assembled parts with many features result in countless possibilities for error that need to be recorded. No problem for the Deep-Learning Inspector, which reliably identifies and evaluates a wide variety of test objects with over 50 different parameters. Even if there are ambiguous possibilities for a good part, i.e. the article fulfils enough different criteria to be qualified as "IO" (“In Order”) and it detects bad parts, even if there are no clear error patterns.

The learning is done by means of teach-in pictures which show concrete IO situations and NIO situations (in order/not in order). An AI is trained from the corresponding image set, with which even complex tests can be performed fully automatically and instantly. The system is constantly evolving as new images are added. To simplify the training process of deep learning algorithms, the system is equipped with a user-friendly interface. A further advantage is that image documentation is automatically created for each product before delivery.

In principle, a control unit always consists of a camera including lighting, mounted on a robot arm. With this unit, feature by feature is then approached, recorded and automatically evaluated using machine learning.

— Edmund Jenner-Braunschmied, CEO of Nordfels GmbH —
Setup consisting of a GigE Vision camera from IDS including lighting, mounted on a robot arm
Setup consisting of a GigE Vision camera from IDS including lighting, mounted on a robot arm

In practice, however, several control units (hand-eye units) are also used within one testing machine. From two units upwards, various teamwork functions can be used to enable the units to work together. In the "Dark-Field Teamwork-Function" one robot-camera-illumination-unit is only responsible for the illumination of the other unit, while the second robot-camera-illumination-unit takes the image. This function is helpful when a feature can be better highlighted with lateral light than with the diffuse incident light illumination that is standard on every control unit.

Another example of a possible teamwork collaboration is the "Free-Sight Teamwork Function". In this case, one robot unit helps the other by using a small stick to hold to the side any cables or hoses of the test object that may be in the field of view of the other camera unit. This allows the second robot camera unit to take the image without disturbance.

With this intelligent, flexible system, end-of-line tests, which are currently often only fulfilled by laborious, exhausting work situations, can be automated in a future-proof way.

Camera

One GigE Vision camera from IDS is used per system. In addition to the interface, the decisive factors for Nordfels in selecting the camera model were size and sensor. The GV-5890SE features the IMX226 rolling shutter CMOS sensor. The 12 megapixel sensor (4000 x 3000 px, pixel size 1.85 µm) from the Sony STARVIS series has exceptional light sensitivity with low noise and a frame rate of 10 fps at full resolution. The power supply via Ethernet allows single cable operation up to 100 meters. Thanks to the sensor's BSI ("back-side-illumination") technology, the camera is predestined for tasks that require perfect results even in low light conditions, so that the EOL inspector achieves good results even without the previously-mentioned "dark field teamwork function".

The control unit takes photographs and evaluates characteristic by characteristic (photographs: Zeidler G):

"The uEye camera is used in a wide variety of applications. Many features can be checked with classic grayscale images, as known from industrial image processing. However, there are also features where the colour information plays an important role. Then the photos are triggered in colour mode. In addition, one camera unit has different lighting colours, so that optimal conditions for image acquisition can always be created," explains Edmund Jenner-Braunschmied. OCR reading and code reading, such as 2D codes or DataMatrix codes, are possible with this structure. OCR recognition is also done with deep learning, whereas code reading is done with classic image processing.

Outlook

The market for machine vision, especially in connection with robotics, is growing inexorably in a wide range of industries. Nordfels is also facing this trend. "Whether handling or testing machines, the combinations and possible applications are ", confirms Jenner. Added to this are the new possibilities offered by deep learning and machine learning. "The result is a technical playing field that offers infinite possibilities, but also requires a great deal of expertise and multidisciplinary skills to develop systems that are ultimately easy to operate in production and run with the highest process reliability". Innovative system integrators and machine builders such as Nordfels and future-oriented camera manufacturers such as IDS are facing these challenges in equal measure.

GigE Vision camera from the uEye SE family

Nordfels GmbH