Interfacing IDS NXT using OPC UA
Integrating IDS NXT cameras into factory automation via OPC UA
A software update provides IDS NXT cameras with another future-oriented component for seamless integration into factory automation. A plugin integrates an OPC UA server into the cameras. Equipped with OPC UA through a software update, they make their AI capabilities available via a "recipe collection" in the network and communicate results directly from the field level to the machine and control level without a detour. OPC UA is therefore the ideal supplement for IDS NXT cameras to make camera configurations and Vision App-based inference tasks in industrial environments accessible in a format that is compatible and readable for OPC UA clients. OPC UA is therefore the ideal Industry 4.0 add-on for your IDS NXT cameras.
This tech tip is a follow-up to "IDS NXT Communication" and deals in particular with the new communication possibilities of IDS NXT cameras via the OPC UA server. We show how you can use saved camera settings, start an inference task and query results via an OPC UA client. You can follow the sample workflow completely with little preparation. All you need is a working IDS NXT camera and a PC in the same network.
OPC UA
OPC UA is a key technology. It unifies the communication between clients and servers. In addition, it determines how devices (servers) describe themselves to clients and which information and services they offer. The uniform exchange across all physical layers and ethernet-based fieldbuses up to the cloud brings an enormous simplification of the engineering effort if data does not have to be read, converted and synchronized via different protocols. This saves time and costs in automation and ensures rapid integration into industrial plants. With the OPC Machine Vision Companion specification, the cross-manufacturer and cross-industry communication technology provides the ideal interface extension for IDS NXT cameras in order to easily use AI-based "recipes" in factory automation.
IDS NXT OPC UA Server Plugin
The OPC UA Server Plugin turns IDS NXT cameras into OPC UA servers that can be addressed by any OPC UA client. Image processing tasks are encapsulated by the plugin in function modules, so-called "recipes". Likewise, necessary and sometimes very deep settings for special applications are summarized and stored in configuration data sets. For these measures, the plugin provides corresponding functions via the REST interface. You can use them very easily via the GUI in the IDS NXT cockpit. Image processing results can be queried by OPC UA clients via the result management or subscribed to as events. General device information, such as name or serial number, is provided via the device information model. The OPC UA communication works like an additional camera interface that is available in the ethernet-based network after starting the OPC UA server.
Downloads
The IDS NXT OPC UA plugin can be found in the download area of the IDS website. To do this, select your camera model and open the "IDS NXT Experience Kit > Vision Apps" category. The plugin is installed like any other vision app via the IDS NXT cockpit.
To use the plugin, IDS NXT OS 1.1 or higher must be installed on the camera. In the rest of this tech tip, we will demonstrate an inference task for which you need the "Classifier" vision app.
Download "IDS NXT OS
Download "OPC UA Plugin
Download "Classifier Vision App"
Instructions
To update the operating system of your camera, read the Quick Start Guide "setup-ids-nxt-rio.pdf", chapter "Updating Firmware".
The plugin documentation explains the functionality of the plugin, how to use it via the IDS NXT cockpit and lists the complete REST API.
IDS NXT Quick Start Guide
OPC UA Server Plugin Documentation
OPC UA Client
IDS NXT cameras can be connected to any OPC UA client via the OPC UA plugin. This can be, for example, proprietary software of a machine control or a management system. With the help of OPC UA development libraries for different programming languages such as C++, C# or Python, you can also implement your own OPC UA client. Since we want to show here how easily and quickly the camera can be integrated into existing OPC UA networks, we use the well-known OPC UA client "UaExpert®" to demonstrate the OPC UA plugin. The full-featured client offers a wide range of functions and a GUI that makes it easy to demonstrate the possibilities of the IDS NXT OPC UA server. It supports OPC UA features like data access, events, and calls of OPC UA methods.
Downloads
The OPC UA client "UaExpert®" is available free of charge after one-time registration on the website of Unified Automation GmbH.
Download "UaExpert®"
Using IDS NXT as OPC UA vision system
Tip: IDS NXT OPC UA Tutorial Video
Our tutorial video "IDS NXT - Setup OPC UA communication" shows the following steps in fast forward and gives you a short insight into the possibilities of the plugin based on the communication with the OPC UA client "UA Expert".
To demonstrate controlling an IDS NXT camera via OPC UA, this tech tip describes an example AI image processing workflow that you can easily reproduce yourself with your IDS NXT camera and a few objects ( ballpoint pen, ruler, eraser, key, etc.). To do this, simply follow the instructions below step by step.
1) Setup inference task
- For this example we use the " Classifier " vision app with the pre-installed neural network (CNN) "ImageNet1000". Install and start the app via the " Vapp Manager " in the IDS NXT cockpit.
- The classifier recognises many objects, such as ballpoint pens, keys, business cards, through the pre-trained ImageNet1000. Place any objects under your IDS NXT camera and setup image capture in freerun mode so that your objects are fully imaged with appropriate settings (colour, sharpness, exposure, etc.).
- Use ROIs for the objects you want to identify.
The inference results of the classification of each image capture are displayed at the ROIs and in the results area with the respective probability.
In our case, a ballpoint pen (68%), a sharpener (100%) and an eraser (98%) are recognised.
Instructions
You will also find documentation on how to use the classifier in the download area for your camera.
Classifier Vision App Documentation
2) Start OPC UA server
- You do not need to program anything to provide OPC communication on the part of the camera. Simply install (1) and start (2) the OPC UA server plugin via the "VAPP Manager" in the IDS NXT cockpit on your IDS NXT camera.
- You can start (1) and stop (2) the OPC UA server via the actions of the plugin. By default, the server is started automatically (3) when the plugin is started.
3) Create OPC UA configuration and recipe
- Creating and saving configurations on the IDS NXT camera via the OPC UA plugin is easy via the IDS NXT cockpit.
- Simply name the current camera settings (configuration name) and the inference task with the classifier (recipe name) and save the configuration and recipe via the corresponding actions. With this, you have already completed the OPC UA setup of the camera via the cockpit.
Instructions
Read chapter "Actions" in the OPC UA Server Plugin Documentation.
4) Establish OPC UA client - server connection
- Install and start the OPC UA client "UaExpert®".
- To connect the OPC UA server of the camera, click on the (Add Server) in the menu bar. Switch to "Advanced" tab.
- Give the connection a "Configuration Name"
- Specify the "Endpoint URL" of the server (e.g. "opc.tcp://169.254.173.231:4840")
- Protocol: opc.tcp
- IP address of the camera
- OPC UA server port: 4840
- Select the desired "Security Policy" (supported are: None, Basic128Rsa15, Basic256, Basic256Sha256)
- Select "Message Security Mode" (None, Sign, Sign&Encrypt)
- Enter the access data for your IDS NXT camera in the "Authentication Settings". (e.g. admin or service user)
- Now you can connect with the OPC UA server: "Connect"
5) Request / edit device information
- To test the server connection, you can navigate through the information model of the camera in the "Address Space" window. For example, if you change the device name in UaExpert®, it will also be updated in the IDS NXT cockpit.
- Furthermore, you can subscribe to status and statistics parameters via the " Data Access View" to track their changes. If, for example, you drag the image counter "ImageTotal" into the window, its value will increase dynamically with each new image captured by the camera.
- To reset the image counter, execute the "Call" in the context menu of the method " ResetAll". The subscribed frame counter then starts counting again at "0".
IDS NXT vision system -- Companion Specification according to OPC UA Machine Vision
In the "Vision System" node in the "Address Space" of the UaExpert® , the standardised attributes of the IDS NXT vision system are displayed, which are implemented in the OPC UA server of the camera according to the Companion Specification "OPC Machine Vision".
This allows you to control the prepared inference task of the camera completely via OPC UA in the further course.
Since version v1.6 of the OPC UA Client "UaExpert®" it is mandatory to enter a number into the corresponding fields for all calls of methods with numeric input arguments. Calls with empty fields are invalid and cause a corresponding error message. Our screenshots and videos were created with an older version!
6) Activate a camera configuration
- First retrieve a list of the configurations stored on the camera with a call to the function GetConfigurationList.
ResultCount shows us "1" available configuration with the InternalId "conf_classifier.json". - Use the function ActivateConfiguration to define the currently valid configuration whose saved parameters are then used in the camera. Set the corresponding InternalId as the input value (Id) for the call.
- You can check the currently active configuration via the node Active Configuration. The Id " conf_classifier.json" should appear there in the attributes.
7) Prepare task (recipe)
We now proceed in the same way with the pre-saved image processing task of the camera.
- A call of the function GetRecipeListFiltered with the input value IsPrepared = 2 (DONTCARE_2) returns "1" stored recipe as ResultCount. In our case the saved recipe with the Id "recipe_classifier_imagenet.json".
- With the function PrepareRecipe you now prepare this recipe as an active image processing task by setting the previously queried Id as the input value "InternalIdIn".
Generally, several recipes can be active at the same time. However, a recipe can only be activated if no other recipe is currently prepared with the same Vision App.
You can check whether a recipe is already active by executing the function GetRecipeListFiltered with the input value IsPrepared = 1 (TRUE_1). If ResultCount!= 0, a recipe is already prepared. Then first use the function UnprepareRecipe.
8) Subscribe to a result event
The camera is now fully prepared via the OPC UA client. Before you execute the camera task, the result event must first be subscribed to.
- To do this, add an EventView...
- and drag 'n drop the camera's VisionSystem into this view.
- Activate the SimpleEvents and ConditionTypes fields to get all the details.
- Confirm the selection via the Apply button.
9) Execute task (recipe) - Evaluate event result
Now we let the camera do its job exactly once.
- To do this, call the method StartSingleJob once.
- A ResultReadyEvent is received from the camera in the EventView .
- In the event details, we can see the inference results of the three specified ROIs (ballpoint, pencil_sharpener, rubber_eraser) with the respective probabilities of classification.
Summary
With digital inputs and outputs, RS-232 and a REST web service, IDS NXT cameras and their AI capabilities can already be used very flexibly in applications. (Read the first part of the tech tip " Interfacing IDS NXT").
Through the OPC UA server, you can now actually transform your IDS NXT inference cameras into industrial-grade vision sensors that can be integrated into any modern factory automation system in just a few steps.
The tech tip illustrates how quickly and easily the standardised attributes of a machine vision system, according to the Companion Specification "OPC Machine Vision", are made available via the IDS NXT cockpit. No prior knowledge or programming is necessary. Simply save prepared camera configurations and AI tasks as recipes in the camera. Thus prepared, you can control and use the capabilities of the IDS NXT vision systems with any OPC UA client. The cameras deliver one event per job with inference results that can be used directly by control systems for subsequent tasks.
Equipped with OPC UA communication, IDS NXT inference cameras easily add the ability to detect and classify objects to your factory automation. Integrating artificial intelligence in industry has never been easier. If you want to know more about OPC UA and the integration in IDS NXT cameras, read our technical article " Inference by recipe". You will find further information in the Knowledge Base. If you have any specific questions, please do not hesitate to contact our system consulting directly.
Tip: IDS NXT webinar-video
The webinar video "Integration of IDS NXT into factory automation" demonstrates communication via the IDS NXT camera interfaces using Python sample code and Jupiter notebooks.