Technical Articles

Redefining the industrial camera

Industrial camera feature sets constantly evolve. Although users can import these new functions themselves, they have no influence on the range of functions. IDS offers a solution: Vision App-based industrial cameras of the IDS NXT series can be extended by self-programmed apps which are also provided as standard-compliant features.

In addition to the firmly integrated features from the manufacturer, the IDS NXT concept also enables the user to provide GenICam-compliant functionalities loaded during runtime via the camera description file.
In addition to the firmly integrated features from the
manufacturer, the IDS NXT concept also enables the user
to provide GenICam-compliant functionalities loaded
during runtime via the camera description file.

Industrial cameras are currently supplied with new features and bug fixes via update packages provided by the manufacturer. This allows users to extend the capabilities of their own cameras that are already in the field, giving them more opportunities for their image processing applications. A so-called "dynamic" UI (User Interface) is required to query the current camera functions.

The GenAPI UI (Generic Application Programming Interface) enables this feature query for a GenICam-compliant industrial camera. In addition to the standard feature list (defined by the "Standard Feature Naming Convention"), GenICam also allows user-defined functions to be integrated into the camera's feature node map. Camera manufacturers use this possibility, for example, to offer special custom features that are not (yet) included in the vision standard. However, it depends on the manufacturer when their cameras get an update and which features it includes.

New intelligent camera class

However, the number of programmable cameras in the machine vision market is increasing. This enables the user to execute self-created functions which previously were performed on PCs or embedded boards, directly in the camera. This camera-side processing covers everything from image data pre-processing and filtering to classical image processing and the use of neural networks using deep learning. Programmable cameras thus offer the user completely new possibilities for image evaluation directly "on-camera". However, unlike standard vision cameras, they rarely have a high-speed data interface such as GigE Vision and are therefore significantly inferior to their vision counterparts in terms of transmission speed. Interactions with functionalities must nevertheless be available for the user or the user software via an API. Until now, the majority of these have been proprietary manufacturer interfaces. Due to their independent operating mode, they are often equipped with direct connections to important industrial protocols such as Profinet, EtherNet/IP or OPC-UA. Despite the newly opened possibilities, these new intelligent cameras cannot replace standard vision industrial cameras. Finally, the user still has to decide himself which cameras to use for his application or whether he even needs the capabilities of both camera classes.

IDS NXT industrial cameras with "Smart GenICam" combine the features of industrial cameras and programmable cameras.
IDS NXT industrial cameras with "Smart GenICam" combine the features of industrial cameras and programmable cameras.

Make one out of two

Thanks to the new IDS NXT industrial cameras, users will be relieved of this decision. To achieve this, IDS has expanded its in-house Standard Vision industrial cameras with the Vision App-based IDS NXT concept. This results in cameras that can load and execute image processing apps programmed by users at runtime in addition to their standard firmware features. This feature extension is made possible by the IDS NXT concept using an IDS-specific "Smart GenICam" app. The highlight of this combination is that configuration, control and results of the Vision App can be used via the camera's XML description file in any GenICam-compliant third-party application, such as HALCON. The devices nevertheless remain fully-fledged standard-compliant industrial cameras that can transfer image data to a host PC at full Gigabit Ethernet speed.

Application scaling

IDS NXT in conjunction with the Smart GenICam App enables the scaling of a GenICam-compliant machine vision application between camera and PC. This means that users can decide, depending on the required performance, whether their application is to be executed classically on a PC workstation, completely independently on the camera, or cooperatively distributed on both devices.

With IDS NXT industrial cameras, user processes can also be executed on-camera. This enables shifting of application parts from the host PC (1) partially (2) or completely (3) into the camera.
With IDS NXT industrial cameras, user processes can also be executed on-camera and made vision-compliant. This enables shifting of application parts and decision processes from the host PC (1) partially (2) or completely (3) into the camera.

There are several reasons for pre-processing images or pre-selecting data on the camera (on the edge), rather than continuously transferring large amounts of data to a PC - such as lower network load, hardware-accelerated processing, less energy consumption, lower system costs, to name just a few.

By using generic transport layers (GenTL) - both externally when handling the camera and within the camera firmware when accessing the sensor - it makes no difference where the image data originates for the application layer. There is no need to change the data interface when the application is moved to the camera. This simplifies the development and validation of a Smart GenICam feature. The application code does not have to be changed or reprogrammed for the execution in the camera. That means that the entire image processing application is first developed classically with the camera as a pure image supplier and an application that is completely executed on a host PC. Because the camera does not have to be changed for the subsequent steps, this simplifies further application development and also saves time and money. The developer now decides which parts he would ideally execute directly on the camera and uploads them to an IDS NXT camera model packed as a Vision App. Its firmware integrates the application code into the functional scope of the camera. Control and function parameters as well as results of the dynamic application part are provided by the Smart GenICam App in a vision-compliant manner via the device-specific XML description file. As a result, the application developer has full control over the new camera functionality.

Advantages of application scaling

IDS NXT moves image processing away from the PC and closer to the image source. Instead of transferring complete image data from high-resolution sensors, conclusions and results can be obtained directly on the camera. This reduces the transmission bandwidth required to transfer necessary data to the subsequent system and enables direct triggering of consecutive processes directly by the camera. In addition to partial scaling, certain applications can also be completely handled by the camera if further processing on a PC is not required. This saves space and costs for optional PC infrastructure. The more an application is shifted to the camera, the more energy-efficient the overall system operates, since the energy consumption of the camera platform is much lower than that of a PC workstation. At this year's Vision 2018, IDS demonstrated complete application scaling on an IDS NXT platform using simple image processing examples such as color and shape sorting systems on a Profinet-capable IDS NXT vegas.

Summary

The Vision App-based IDS NXT concept makes image acquisition and image processing available vision conform "on the edge". These new camera devices no longer only work as pure image suppliers, but are also able to work completely independently or support a connected PC with individual pre-processing. Even complex inference tasks with pre-trained neural networks are possible. With the Smart GenICam Vision App, it is also very easy for a customer to make his camera processing tasks available for every GenICam-compliant application.