Interfacing IDS NXT using OPC UA

Integrating IDS NXT cameras into factory automation via OPC UA

A software update provides IDS NXT cameras with another future-oriented component for seamless integration into factory automation. A plugin integrates an OPC UA server into the cameras. Equipped with OPC UA through a software update, they make their AI capabilities available via a "recipe collection" in the network and communicate results directly from the field level to the machine and control level without a detour. OPC UA is therefore the ideal supplement for IDS NXT cameras to make camera configurations and Vision App-based inference tasks in industrial environments accessible in a format that is compatible and readable for OPC UA clients. OPC UA is therefore the ideal Industry 4.0 add-on for your IDS NXT cameras.


OPC UA is a key technology. It unifies the communication between clients and servers. In addition, it determines how devices (servers) describe themselves to clients and which information and services they offer. The uniform exchange across all physical layers and ethernet-based fieldbuses up to the cloud brings an enormous simplification of the engineering effort if data does not have to be read, converted and synchronized via different protocols. This saves time and costs in automation and ensures rapid integration into industrial plants. With the OPC Machine Vision Companion specification, the cross-manufacturer and cross-industry communication technology provides the ideal interface extension for IDS NXT cameras in order to easily use AI-based "recipes" in factory automation.

IDS NXT OPC UA Server Plugin

The OPC UA Server Plugin turns IDS NXT cameras into OPC UA servers that can be addressed by any OPC UA client. Image processing tasks are encapsulated by the plugin in function modules, so-called "recipes". Likewise, necessary and sometimes very deep settings for special applications are summarized and stored in configuration data sets. For these measures, the plugin provides corresponding functions via the REST interface. You can use them very easily via the GUI in the IDS NXT cockpit. Image processing results can be queried by OPC UA clients via the result management or subscribed to as events. General device information, such as name or serial number, is provided via the device information model. The OPC UA communication works like an additional camera interface that is available in the ethernet-based network after starting the OPC UA server.

OPC UA Client

IDS NXT cameras can be connected to any OPC UA client via the OPC UA plugin. This can be, for example, proprietary software of a machine control or a management system. With the help of OPC UA development libraries for different programming languages such as C++, C# or Python, you can also implement your own OPC UA client. Since we want to show here how easily and quickly the camera can be integrated into existing OPC UA networks, we use the well-known OPC UA client "UaExpert®" to demonstrate the OPC UA plugin. The full-featured client offers a wide range of functions and a GUI that makes it easy to demonstrate the possibilities of the IDS NXT OPC UA server. It supports OPC UA features like data access, events, and calls of OPC UA methods.

Using IDS NXT as OPC UA vision system

To demonstrate controlling an IDS NXT camera via OPC UA, this tech tip describes an example AI image processing workflow that you can easily reproduce yourself with your IDS NXT camera and a few objects ( ballpoint pen, ruler, eraser, key, etc.). To do this, simply follow the instructions below step by step.

1) Setup inference task

  • For this example we use the " Classifier " vision app with the pre-installed neural network (CNN) "ImageNet1000". Install and start the app via the " Vapp Manager " in the IDS NXT cockpit.
  • The classifier recognises many objects, such as ballpoint pens, keys, business cards, through the pre-trained ImageNet1000. Place any objects under your IDS NXT camera and setup image capture in freerun mode so that your objects are fully imaged with appropriate settings (colour, sharpness, exposure, etc.).
  • Use ROIs for the objects you want to identify.
    The inference results of the classification of each image capture are displayed at the ROIs and in the results area with the respective probability.
    In our case, a ballpoint pen (68%), a sharpener (100%) and an eraser (98%) are recognised.

2) Start OPC UA server

  • You do not need to program anything to provide OPC communication on the part of the camera. Simply install (1) and start (2) the OPC UA server plugin via the "VAPP Manager" in the IDS NXT cockpit on your IDS NXT camera.
  • You can start (1) and stop (2) the OPC UA server via the actions of the plugin. By default, the server is started automatically (3) when the plugin is started.

3) Create OPC UA configuration and recipe

  • Creating and saving configurations on the IDS NXT camera via the OPC UA plugin is easy via the IDS NXT cockpit.
  • Simply name the current camera settings (configuration name) and the inference task with the classifier (recipe name) and save the configuration and recipe via the corresponding actions. With this, you have already completed the OPC UA setup of the camera via the cockpit.

4) Establish OPC UA client - server connection

  • Install and start the OPC UA client "UaExpert®".
  • To connect the OPC UA server of the camera, click on the (Add Server) in the menu bar. Switch to "Advanced" tab.
  • Give the connection a "Configuration Name"
  • Specify the "Endpoint URL" of the server (e.g. "opc.tcp://")
    • Protocol: opc.tcp
    • IP address of the camera
    • OPC UA server port: 4840
  • Select the desired "Security Policy" (supported are: None, Basic128Rsa15, Basic256, Basic256Sha256)
  • Select "Message Security Mode" (None, Sign, Sign&Encrypt)
  • Enter the access data for your IDS NXT camera in the "Authentication Settings". (e.g. admin or service user)
  • Now you can connect with the OPC UA server: "Connect"

5) Request / edit device information

  • To test the server connection, you can navigate through the information model of the camera in the "Address Space" window. For example, if you change the device name in UaExpert®, it will also be updated in the IDS NXT cockpit.
  • Furthermore, you can subscribe to status and statistics parameters via the " Data Access View" to track their changes. If, for example, you drag the image counter "ImageTotal" into the window, its value will increase dynamically with each new image captured by the camera.
  • To reset the image counter, execute the "Call" in the context menu of the method " ResetAll". The subscribed frame counter then starts counting again at "0".

6) Activate a camera configuration

  • First retrieve a list of the configurations stored on the camera with a call to the function GetConfigurationList.
    ResultCount shows us "1" available configuration with the InternalId "conf_classifier.json".
  • Use the function ActivateConfiguration to define the currently valid configuration whose saved parameters are then used in the camera. Set the corresponding InternalId as the input value (Id) for the call.
  • You can check the currently active configuration via the node Active Configuration. The Id " conf_classifier.json" should appear there in the attributes.

7) Prepare task (recipe)

We now proceed in the same way with the pre-saved image processing task of the camera.

  • A call of the function GetRecipeListFiltered with the input value IsPrepared = 2 (DONTCARE_2) returns "1" stored recipe as ResultCount. In our case the saved recipe with the Id "recipe_classifier_imagenet.json".
  • With the function PrepareRecipe you now prepare this recipe as an active image processing task by setting the previously queried Id as the input value "InternalIdIn".

8) Subscribe to a result event

The camera is now fully prepared via the OPC UA client. Before you execute the camera task, the result event must first be subscribed to.

  • To do this, add an EventView...
  • and drag 'n drop the camera's VisionSystem into this view.
  • Activate the SimpleEvents and ConditionTypes fields to get all the details.
  • Confirm the selection via the Apply button.

9) Execute task (recipe) - Evaluate event result

Now we let the camera do its job exactly once.

  • To do this, call the method StartSingleJob once.
  • A ResultReadyEvent is received from the camera in the EventView .
  • In the event details, we can see the inference results of the three specified ROIs (ballpoint, pencil_sharpener, rubber_eraser) with the respective probabilities of classification.