All new features at a glance
Highlights of the IDS NXT product releases
IDS NXT malibu - On-Camera AI + Compression + Streaming!
New: 4K streaming in perfect image quality directly from the camera
Having launched the IDS NXT malibu series with the onsemi AR0521 sensor, IDS is now adding to the range: The camera is now also available with the Sony Starvis 2 sensor IMX678 for streaming in 4K! This means even more detail and even better image quality - and extends the scope of applications, especially in the low-light range.
Consumer technology meets industrial expertise: There is more packed in there!
A new class of intelligent industrial cameras has emerged from the collaboration of IDS and Ambarella: IDS NXT malibu combines perfect image quality + On-Camera AI + video compression + streaming. All this is performed directly in and output from the camera.
IDS is the first company to offer this special chip - previously known from action cameras - for use in industrial applications. An integrated ISP with helpful automatic features, such as for brightness, noise, and colour correction, draws the optimum image quality. Look forward to impressive images and videos - and even more. For further information on new software highlights relating to IDS NXT malibu, please refer to the Release 4.1 tab.
AI and ISP "on the edge": Evaluations & Visualisations without a PC
The camera operates as an "Edge Device" and enables compressed video streams (H264) with full sensor resolution at high speed (> 25 fps) and their output on end devices via RTSP protocol.
In addition, the camera live images can be displayed for visualisation per MJPEG stream directly in any web browser (without AI overlay) - via HTTP and not requiring any additional software!
"With IDS NXT malibu, we have developed a camera series that analyses images in real-time and can integrate results directly. This makes it suitable, for example, for applications that require a live image with AI overlay," explains Kai Hartmann, Product Innovation Manager at IDS.
What opportunities do the intelligent industrial cameras open up for your applications? Image processing at the cutting edge of technology: discover the possibilities of the IDS NXT system.
IDS NXT 4.1: Comprehensive support for IDS NXT malibu!
MJPEG as an alternative to streaming via RTSP
The new MJPEG stream provides you with an easy-to-use visualisation option for both malibu models in FullHD (sensors: onsemi AR0521 and Sony IMX678). The stream uses HTTP and can be displayed live in any browser without additional plug-ins. If you also want to visualise AI overlays in the live image, opt for streaming via RTSP. This utilises the entire sensor resolution - for the IMX678, this means streaming in 4K at 30 fps!
Classify, detect objects and identify anomalies with user-trained CNNs
In conjunction with the IDS NXT 4.1 software release, neural networks can now be trained with the IDS lighthouse AI vision studio for the Ambarella SoC of the IDS NXT malibu camera family. You can now also create and execute vision apps for classification, object recognition and anomaly detection for the camera platform according to your specifications.
IDS NXT 3.2 - Optimised user guidance through new project view
The free update improves usability with new project guidance in IDS lighthouse as well as with a redesigned menu and easier access to results in the block-based editor.
Project view
The new user interface allows to manage related neural networks, image datasets and vision apps in one project. The workflow view takes the user sequentially through all phases of application development.
State-dependent icons and hints improve the transparency of all process steps. So it is always immediately transparent where user interaction is still required or which components have already been successfully configured.
IDS lighthouse thus now supports users even better in creating their own AI-based applications.
You find the release notes directly in IDS lighthouse.
IDS NXT 3.1 - OCR and finding objects via Block-based Editor
The update expands the system's functionality to include rule-based image processing such as OCR and finding objects without trained CNN. In addition, customers benefit from workflow improvements and easier integration via OPC UA.
Recognise texts
The update makes the image processing function OCR, short for "optical character recognition", available in the block-based editor. With this function, German and English characters, words or entire lines of text on labels, business cards and other documents can be found and read out in the image. Subsequently, certain values or properties can be retrieved for them.
Finding objects without AI
The AI function Object Detection has been available in IDS NXT for a long time. But what if objects need to be found without first training a neural network? This has now become possible in the block-based editor. They are identified via brightness values. Afterwards, they can be filtered or sorted out.
Better visualisation via attention maps
You can now create a visualisation for each test data set based on attention maps in the AI Vision Studio IDS lighthouse. This allows you to check the inference with your own images, visually reconstruct AI decisions and thereby assess the quality of the trained neural network.
Simplified OPC-UA access
In addition to the standardised Companion Specification for machine vision systems, we also provide our own easy-to-understand information model for accessing IDS NXT via OPC-UA. We have extended this with IDS NXT 3.1 to make integration via OPC-UA even easier for applications such as factory automation.
IDS NXT 3.0 - New AI method for anomalies
With the software release IDS NXT 3.0, we are expanding the application possibilities for the all-in-one AI system to include, among other things, an additional deep learning method and also ensure better traceability in the training of neural networks.
Anomaly Detection
You can now use IDS NXT cameras for anomaly detection in addition to object detection and classification. Only a small amount of training data is required to reliably identify both known and unknown process deviations. This reduces the time and effort required to create and annotate the training data.
Code Reader
With the new code reader function in the block-based editor, you can quickly and easily enable IDS NXT cameras to locate, identify and read different types of code and the required parameters.
Transparency in the training process
As early as during the training and testing process in IDS lighthouse, attention maps show which image areas have an impact on classification results. This means that training errors or biases can be detected and eliminated at an early stage, before the AI is working in the camera. This reduces review and maintenance loops in the development process and therefore accelerates commissioning.
IDS NXT 2.6 - App your camera!
This software release focuses on simplifying the creation of apps. Starting a development project is often one of the biggest challenges in realising a project. With the help of the new Use Case Assistant in IDS lighthouse, you can configure a complete vision app under guidance in just a few steps, which you can then run directly on an IDS NXT camera. Another new feature is the Block-based Editor, which allows you to develop your own programme sequences with AI image processing functions such as object recognition or classification without any programming knowledge.
Use Case Assistant
In the Use Case Assistant, you select the use case that suits your project. With queries and tips, the wizard guides you through the process of creating the Vision App project and creates the programme code, just like in an interview. In the process, it links existing training projects with the Vision App project or creates new training projects and data sets in IDS lighthouse if required.
Block-based Editor
With the modular blocks and the intuitive user interface, anyone can realise their own projects using AI-based image processing (such as object detection or classification) in the form of an individual Vision App without having to know the syntax of a specific programming language. In this case, you create the Vision App project yourself and use the predefined blocks of the code editor to build your Vision App graphically yourself, including processes such as loops and conditional statements.
Confusion Matrix
In the "Training Sessions" area of the "Training" tab of IDS lighthouse, you can display a confusion matrix for classification networks that you have trained from version 2.6. The confusion matrix is suitable for assessing the performance of your classification network and shows how the test or validation data is classified compared to the assigned labels.
Multi-camera support in IDS NXT cockpit
You can now monitor camera status, images and results for multiple cameras on one screen, and keep up to date with your applications. In addition, the configuration including login can be saved for quick access.
IDS NXT 2.0 - AI for Factory Automation
The update expands the AI capabilities of IDS NXT with object recognition, introduces the OPC UA interface and, thanks to a new development environment, ensures that custom image processing tasks can be programmed as vision apps and executed on the cameras.
New development environment for professionals
Developers and AI specialists can use the "IDS NXT Vision App Creator" to dive even deeper into the system, create individual vision apps and thereby adapt IDS NXT cameras even better to their own needs. This makes even the combination of classifications or object recognition with classic image processing, such as measuring, feasible in one device.
Automation via OPC UA
IDS NXT rio and rome cameras can now communicate with machines and controllers in the same language and exchange tasks or results directly. The use of OPC UA enables the direct integration of these intelligent cameras into factory automation, without additional hardware or programming effort. The standardised communication also creates the basis for the "plug-and-produce" of new devices. This shortens the commissioning of machine vision systems to a minimum.
Video platform dedicated to cameras and applications
On the IDS Vision Channel, we regularly introduce you to new product features, guide you through important workflows with tutorials and explain which technology can be used for which applications. With your IDS website account, you can participate in live events as well as use the convenient media library.