Technical Articles

With (3D) camera technology to zero-defect production?

Software-supported camera image evaluation can assist in early detection of defects and irregularities in production processes. This early defect detection thus constitutes a significant contribution to quality assurance. Consequently, more and more production lines are equipped with state-of-the art camera technology, in an attempt to get closer to zero-defect production.

Given this objective of improving product and process quality, manufacturing procedures are in a state of constant change. Driven to accelerate throughput and reduce waste by the powerful pressures of time and cost, a great deal of energy is invested in automation of the test steps. Humans cannot hope to achieve these objectives unaided by machines. As more and more quality assurance functions are automated, essential but time- and cost-intensive lab tests are being replaced by "inline at the shop floor" product inspections. In this scenario, modern camera technology maintains an overview of the test objects and their properties at all times and in almost any location as production progresses, without actively intervening in the train of events. In conjunction with image processing, the use of camera images offers a wide variety of advantages and options for use. "Optical measurement and test procedures" enable lightning-fast quality tests to be carried out during the manufacturing process, and they involve absolutely no wear. When industrial cameras are used in place of simple barcode readers, not only the content of the code can be read, but the quality of the code, its alignment, color, size, surroundings and much more can be determined and evaluated. This not only expands test capabilities, but also the test quality and also makes the further automation of quality assurance easier.

Cameras can provide early detection of defects that have occurred, before they cause aftereffects that may be difficult if not impossible to correct. But what camera technology is capable of meeting the stringent requirements applicable to measurement and test equipment and enabling machines to make autonomous decisions, thereby providing humans with the best possible assistance, or even replacing them?

"With 3D image processing, the flexibility and accuracy of measurement and test processes can be improved immensely“, Dr. Martin Hennemann, Product Manager for Ensenso at IDS Imaging Development Systems GmbH

Modern 3D camera technology enables robots to analyse their environment so that they can respond to different situations autonomously. Besides the dimensions and position of objects in the shop floor, this data also enables exact conclusions to be drawn regarding deviations or imperfections with respect to reference objects. This is made possible by inline inspections, which show whether the "correct" product components are being assembled.

A high potential for error exists precisely in the automotive industry, in which production involves a enormous number of variants. Customers can assemble their perfect car according to their individual specifications. The choices include different tires, engine variants, chassis or interior equipment. So production personnel benefit particularly from automated systems which sound the alarm in good time, before a wrong or defective part is installed. This also reduces the labour and time required for inspections after the fact, or – which would be much worse – the risk of production downtimes due to errors.

"The high degree of individualization in the configuration of cars inevitably results in an immense range of variants, which is practically impossible to manage and check without the assistance of camera technology“, Dr. Martin Hennemann

With HALCON 3D image processing, even the tiniest differences between the object and the reference model can be detected. Image: Demo by IDS, Control 2018, captured with an Ensenso 3D stereovision camera.
With HALCON 3D image processing, even the tiniest differences between the object and the reference model can be detected. Image: Demo by IDS, Control 2018, captured with an Ensenso 3D stereovision camera.

IDS demonstrated an example of "3D object verification" in April 2018 at the "Control 2018" international trade fair for quality assurance in Stuttgart. The demo system reconstructs objects with an Ensenso N35 3D camera and then performs a comparison of data with the aid of HALCON image processing. For this, the 3D data generated is compared with a reference model which enables deviations and imperfections to be detected. The 3D image processing identifies irregularities in test objects which are not even visible to the human eye. The quality of manufactured objects can be considerably improved for subsequent process steps by ultrahigh precision digitization with 3D cameras by Ensenso followed by image processing for object verification.

"3D complements 2D camera technology. Each technology has its own advantages. The combination of 2D and 3D performs the tasks of today and tomorrow while effectively minimizing production errors“, Dr. Martin Hennemann

For Dr. Martin Hennemann, 3D technology is an advantageous supplement to 2D camera technology. In combination, they can be used to perform an ever growing range of functions. Each of the two technologies has its advantages for certain applications. With 3D data, it is possible to determine shapes, surface structures, or the position of objects in space. 2D data enables edges and colours to be detected with precision and markings to be read. In the case of the IDS control demo, the 3D data is used to perform an object comparison. Any part codes present can also aid identification. They are ideally decoded with 2D camera images. Car manufacturers also use camera-based inspection methods with combined 3D and 2D technology for final inspection, in order to find clearance dimensions on a finished car body and precisely test or measure it against specifications. Camera technology in general offers many possible ways to get closer to the ideal of zero defect production. In order to effectively minimize production errors, one or the other technology, or even a combination of the two may then be necessary, depending on the test criterion to be applied.

For text recognition (OCR) and edge detection, classic 2D image material is the best source material for image processing.
For text recognition (OCR) and edge detection, classic 2D image material is the best source material for image processing. Source: MVTec HALCON

Camera systems also support manual processes in manufacturing. They monitor manual workflows at manual workstations. The image evaluation checks whether the operator picks up the right parts, the correct number of screws are fitted, or if a part has been forgotten. A downstream system indicates whether operations have been performed correctly or incorrectly by means of audible or visual alerts on monitors or by projection. This ensures a quality control component during production.

Camera and production technologies are both undergoing constant development. But each new camera technology does not immediately replace the existing one. Having progressed immensely in the last few years, the new 3D technology is able to display different features of an object. It has made new applications possible and serves as a supplement to 2D camera technology, but it does not replace it. Cameras are also evolving constantly, from "simple" image delivery machines to "smart" adaptable image processing solutions. With the right combination of camera, sensors, intelligence, communication and versatility, machine-vision systems such as the IDS NXT platform not only offer the functions of 2D and 3D cameras but also represent a significant contribution to error prevention in quality assurance.

Today, as ever before, "perfect" production without reworking or defects is the ideal aspiration, which is unattainable even at immense expense and with the latest technology. However, at the same time the use of camera and evaluation technology has now become indispensable for any organisation intending to take a significant step towards the goal of zero defect production.

Video: 3D object verification with Ensenso and HALCON for Embedded Vision