Ensenso XR Onboard processing

Embedded 3D vision

High-resolution 3D data directly from the camera

If large volumes or several object views are to be checked automatically by 3D cameras, e.g. on constantly running production lines in the automotive industry, high-resolution 3D result data must be generated and processed quickly. Stereo camera systems with large 5 MP sensors and variable baselines provide the ideal output data. However, interfaces and CPU power can quickly form the bottleneck in such high-performance 3D applications.

The challenge: to reduce data rates and performance requirements on system components without compromising data quality. At the same time, the systems should be space-saving and efficient. Embedded 3D cameras of the Ensenso XR series with integrated data processing are therefore the next logical step.

In machine vision applications with 3D cameras that work according to the principle of spatial vision (stereo vision), camera images are processed with high resolution and frame rates in order to make result data available to further processes as quickly as possible. The calculation of the three-dimensional data, so-called "point clouds", from the image material of the stereo cameras requires several complex process steps, which were previously performed by powerful industrial PCs (IPCs). With increasing demands regarding the quality and speed of these result data, modern 3D stereo cameras, such as the Ensenso X series, use high-resolution 2D cameras with Gigabit Ethernet interfaces. However, the transmission of the 2D output data to the processing IPC requires optimally utilizing the network bandwidth in order to avoid time delays or data loss. In addition to that, the processing power of the IPC hardware must constantly increase in order not to restrict the overall system.

By using high-quality components, the performance of such 3D camera systems can be further increased. Thanks to exchangeable 2D cameras, the flexible design of the Ensenso X series is not tied to specific data interfaces and sensor resolutions and can continue to grow with the requirements for speed, object sizes and quality. But high-resolution, fast GigE cameras, specially shielded cables, high-performance network technology and powerful PC hardware are simply too expensive for some applications. In addition, sufficient space must be available for these peripherals.

With the new XR camera series, Ensenso is pursuing a different approach. According to the principle of "Internet of Things" (IoT), each individual component in a "distributed system" carries out a specific task and produces results that can be used directly by other systems. In the case of a 3D camera, these are three-dimensional coordinates of pixels of a real object.

Onboard 3D processing

A SoC (System-on-Chip) integrated into the Ensenso XR projector unit allows the camera to execute the 3D processes itself, including stereo analysis. After correction of the lens distortion, the 2D output images are transferred to an axis-parallel stereo system by a virtual rotation of the cameras (rectification), which greatly simplifies all subsequent analyses. Then, the highly optimized matching algorithms for static or moving scenes search the recorded image pairs for corresponding pixels. Due to the different perspectives of the cameras, these pixels are also subject to different horizontal shifts in the image plane, known as "disparity". Due to the geometric relationships in the parallel stereo system, this disparity represents a measure of the spatial depth of a 3D point in millimetres after the application of beam sets and knowledge of known system parameters such as focal lengths, pixel sizes and the base length of the stereo system.

These time- and computation-intensive pixel operations are performed in parallel by a supporting FPGA in the camera. This allows a 3D data rate to be realized which is comparable to that of an Ensenso X system which performs stereo analysis on a desktop PC with an Intel Core i7 Quad CPU.

"By combining integrated data processing with FlexView2 technology, accurate and fast acquisition of image details goes hand in hand"

— Martin Hennemann, Product Manager and 3D image processing specialist at IDS —

New independence

The higher autonomy of the Ensenso XR stereo camera compared to other 3D cameras becomes an important selection criterion for 3D applications not only in terms of speed. Reduced performance requirements on network peripherals and IPC hardware simplify the entire construction of a 3D application and reduce costs, especially in multi-camera systems.

Furthermore, the new Ensenso XR projector unit has an integrated front light. When in use, it supports the calibration of the working environment or improves the image quality of the 2D camera images if the ambient light is insufficient or no external lighting is available. Besides the GigE data connection, an additional WiFi interface is planned, which will be very useful for temporary access to data and parameters when wiring is difficult or cost-intensive.

The many improvements in the Ensenso XR series open up new fields of application for 3D camera technology. Integrated data processing is the next logical step in optimizing data exchange.