Interfacing IDS NXT

How to integrate IDS NXT cameras in your factory automation

With IDS NXT we have created a new generation of vision systems for industrial applications which can be used as powerful vision sensors in factory automation. Also, running IDS NXT Experience Kit in conjunction with Deep Learning is particularly easy to use. The all-in-one solution provides everything needed to create powerful inference cameras by running individual neural networks directly on-the-edge. From acquiring and managing images, to training the AI, to the final working IDS NXT inference camera, it's done in only a few steps. But how are the inference cameras integrated into the own environment?

In this Tech Tip we particularly address the interfaces for integrating IDS NXT cameras. And we show comprehensibly how you can use inference results in your own environment.

IDS NXT Interfaces

As compact embedded vision systems, IDS NXT cameras can handle image processing tasks directly on-the-edge and provide results without additional PCs. As a result, they can be used as autonomous vision sensors. In terms of access and data exchange requirements for external systems, they differ fundamentally from standard industrial cameras, which mainly serve as image providers for PC-based image processing. Therefore, we have equipped our IDS NXT cameras with interfaces that are easier and more flexible to use in IoT-type application scenarios.

IDS NXT cockpit
The IDS NXT cockpit provides access to all important settings and functions of IDS NXT industrial cameras and is completely based on their REST interface.
Additionally, you can install or configure Vision Apps. No matter which interface you use to work with the camera later on - the IDS NXT cockpit features an easy-to-use GUI to set up camera applications comfortably and configure them for runtime operation.

IO and RS-232

Even in the age of Industry 4.0, digital inputs/outputs and UART-based serial interfaces and the fieldbuses based on them offer numerous advantages over their state-of-the-art successors. Even for complex part inspections and image-based verifications, the supposedly simple results IO or NIO are usually sufficient to quickly and easily control products in large and modern industrial lines. Hardly any other interface can be implemented as easily and inexpensively as digital inputs or outputs. Similarly, serial device communication with RS-232 is supported by so many manufacturers and users. For no other is there a more comprehensive range of proven accessories. This is why RS-232 is still one of the standards in industrial communication today.

With IO and RS-232 interfaces, IDS NXT cameras complement many existing systems with compact and powerful embedded vision solutions without having to change the existing infrastructure.

IDS NXT REST Interface

With a REST (Representational State Transfer) web service, IDS NXT cameras offer a standardized open interface for machine-to-machine communication that allows easy networking. This enables parameters and results of the cameras to be changed or queried. The REST web service is based on the HTTP protocol and is therefore available in most networks.

The standard HTTP methods such as GET, POST, PUT, PATCH, etc. are used as a uniform interface. This means that the IDS NXT REST interface is not only platform-independent, but also, due to the widespread use of HTTP methods, can be used in a wide variety of clients (PCs, smartphones, tablets). While C++ applications require a completely set up development environment, REST(ful) APIs can be served by web-enabled device classes without additional software or gateways. This not only makes it easier to get started, but also makes it possible to package extensive setup actions into simple scripts.
For generic communication with autonomously operating vision sensors, REST can therefore be used in the world of IoT (Internet of Things) as well as IIoT (Industrial Internet of Things) more easily and with less overhead than a PC-based interface, such as GenICam. For IDS NXT cameras, REST is not only the "runtime" interface for data and result transmission, but is also used for the configuration (e.g. IDS NXT cockpit) of the camera, its apps and interfaces (IO, RS232).

Why REST?

  • Stateless
    Stateless communication ensures that each client-server message is self-contained and includes all necessary information about the current application status. This means that the REST service does not need to store anything between executions. Session management in the server is therefore not necessary, allowing the Web service to be scaled as required in the network. This makes REST ideal for use in IoT.
  • Unified and platform-independent
    With REST, systems offer a standardized open interface based on web technology that allows for easy networking. Data and results can be exchanged between all (I)IoT devices without protocol conversion or gateways.
  • Easy and resource-saving
    Building, passing, and possibly processing the results of simple REST API calls, in most cases, requires little processing power. Moreover, due to the widespread use of standard HTTP methods, the necessary technical requirements for REST communication can already be used on most web-enabled devices without additional software.
  • Location-independent
    Clients and servers can be operated separately ("distributed system"). Even if they are hosted by different servers, communication via REST is still possible.
  • Industry and web-compatible
    Easy access and transfer of data is an essential part of IIoT. With its flexibility and ability to use widely available resources, REST can also simplify industrial automation. Many systems and devices can be integrated quickly and easily by using cross-application and cross-device communication. Web-compatibility is a door-opener for a more comprehensive integration strategy. The evolution and development of the industry also benefits from the openness to new systems and technologies.

The IDS NXT REST web service is implemented with the following capabilities:

  • The REST web service is always active during regular operation.
  • Access protection is implemented via HTTP Basic Authentication. This means each request requires the user data of an authorized user for authentication, which is then transmitted Base64-encoded.
  • The responses are transmitted in JSON or XML format.

IDS NXT communication via IOs and RS-232

In order to communicate camera image processing results via digital inputs/outputs or the serial interface, you simply need to start the two IDS NXT Vision Apps "GPIO Gateway" and "RS-232 Gateway" via the IDS NXT cockpit and set from which vision app you want to output specific results. No programming is required. All settings can be done via the GUI.

As an example, you can configure the digital outputs to be active for a certain time each time the camera AI (CNN manager with "ImageNet1000") has classified specific objects (e.g. "Good" or "Bad"). Or you can communicate the detected object class and the probability of inference as result values over the serial connection.

IDS NXT REST Communication

The big advantage of REST for developers is its web-compatibility and thus the simplicity of the required tools. Therefore, we show examples how you can easily use the HTTP methods GET, POST, PUT and DELETE to retrieve data from an IDS NXT camera.

Please note that this tech tip is not intended to be a comprehensive documentation for using the REST API. We would rather like to give you basic hints to make your start easier. The "rest" is up to you!

To be able to reproduce the following examples, you need an IDS NXT camera with access via a Windows PC in the same network. To avoid stupid queries of parameters, we will let the camera do something useful. We use the camera AI and the pre-installed CNN "KritzelNN". This enables the camera AI to recognize drawings of people, dogs, cats and cameras. Via the REST-API we request what it recognizes on a self-painted drawings.

Test scenario "I see something you don't."

  • Network with IDS NXT rio or rome camera and a Windows PC
  • Running vision app "CNN Manager" with the pre-installed neural network "KritzelNN".
  • Various drawings of a person, a camera, a cat or a dog.

REST call with "cURL" (command line)

cURL is a command line tool for transferring data via URLs, which you can simply run at the command prompt (also cmd.exe). To query the inference result (KritzelNN) of the last camera image, we send a 'GET' request with the desired resource '/vapps/cnnmanager/resultsources/last'.

Camera image:

C:\Users\ids>curl -sX GET http://192.168.2.102/vapps/cnnmanager/resultsources/last --user admin:ids
{
    "inference": {
        "Top1": "person",
        "Top2": "camera",
        "Top3": "cat",
        "Top4": "dog"
    },
    "inference_propability": {
        "Top1": "0.99",
        "Top2": "0.01",
        "Top3": "0.00",
        "Top4": "0.00"
    },
    "inferencetime": {
        "Content": "43"
    }
}
C:\Users\ids>curl -sX GET http://192.168.2.102/vapps/cnnmanager/resultsources/last --user admin:ids | jq ".inference.Top1, .inference_propability.Top1"
"person"
"0.95"

Now place various drawings in front of the camera lens for testing and call up the respective inference results.

REST call in the browser

Directly via the address line

The inference query can also be very easily done via web browser (e.g. Firefox or Google Chrome). To do this, enter the following URL in the address line:
http://admin:ids@192.168.188.21/vapps/cnnmanager/resultsources/last
The user credentials are transferred in the standard HTTP authorization header.

Unfortunately, for security reasons not all browsers (e.g. IE) support embedded credentials in the URL anymore!

With Browser Plugins

Rest API Browser Plugins
There are also a number of useful browser plugins with very clear GUIs for testing REST calls.

Comfortable GUI interface for REST communication directly in the browser
Comfortable GUI interface for REST communication directly in the browser
Postman is a development and test environment for Rest APIs
Postman is a development and test environment for Rest APIs

Data that you repeatedly need can thus be conveniently provided in variables. But probably the most important function of Postman is its code generation. At the push of a button, the client creates fully usable code snippets from your tested and saved REST calls for many common programming languages and other clients, such as cURL, C, C#, Python, Java or Swift.

The Postman code snippets provide you with a good base for integrating IDS NXT cameras into self-programmed applications.

REST API integration into programmed applications

Although no development SDK is provided for IDS NXT cameras, for e.g. C++ applications, this does not mean that a programmatic integration of the camera is not possible. Using REST or HTTP connections is possible in most programming languages without additional software from the manufacturer. That is why a REST client like Postman is able to provide you with fully functional code snippets for direct use.

Using our AI camera result as an example, we show you how to program the REST call in Python.

REST call with Python

import http.client
import mimetypes
conn = http.client.HTTPSConnection("192.168.188.21")
payload = ''
headers = {
  'Authorization': 'Basic YWRtaW46aWRz'
}
conn.request("GET", "/vapps/cnnmanager/resultsources/last", payload, headers)
res = conn.getresponse()
data = res.read()
# output camera response
print(data.decode("utf-8"))

Python is a very common programming language in IoT. We would therefore like to explain the integration of our IDS NXT cameras into a production automation system in more detail using Python.

1. Log-in and image transfer (connect_and_get_image.ipynb)
The first example shows how to use the GET command to authorize and retrieve an image from your IDS NXT camera.

2. Change exposure time (change_exposure.ipynb)
The second example shows how to request and modify camera parameters. To do this, we change the exposure value using the PATCH command.

3. Trigger image acquisition (trigger_image_acquisition.ipynb)
Using the POST command, we will show in the third example how to trigger an image capture to retrieve the image content of a modified scene.

4. Configure the IO Gateway (set_gpio_gateway.ipynb)
In the fourth example we will show you how to toggle the cameras digital outputs depending on the results of the CNN manager vision app.