Picking the unknown

Robust and flexible robotic automation in logistics

The global pandemic has radically impacted the supply chain and logistics industry, making the need for robotic automation more urgent than ever. With more than 70% of labor in warehousing being dedicated to picking and packing, numerous companies are gradually investing in logistics automation. But what happens when the robots must handle an unlimited number of (unknown) stock keeping units? These companies need a fast, reliable, and robust way to automate picking and placing of a large variety of objects.

This challenge was taken up successfully by the Dutch company Fizyr. The computer vision company based in Delft focuses on enabling robots to pick unknown objects even in harsh logistics environments. The result is an automated vision solution that enables logistic automation in various conditions and applications, like item picking, parcel handling, depalletising, truck unloading or baggage handling. To complete the system with the optimal hardware, Fizyr integrates compact, robust Ensenso 3D cameras in combination with high performance GigE uEye cameras from IDS.

Application

Instead of providing a proprietary picking cell, like most players in the market do, Fizyr has created a plug-and-play modular software product that integrates smoothly with any system, giving integrators the freedom to choose the best hardware (e.g. Cobots or Industrial robots) for their picking cell. While facilitating cost, robustness, and speed optimization, IDS is the preferred partner in terms of cameras. In essence, the camera sees products to be picked or classified. Depending on the individual customer application, up to four Ensenso 3D cameras in combination with powerful GigE uEye CMOS cameras are used.

Ensenso 3D camera allows robots individual handling of unknown objects
Ensenso 3D camera allows robots individual handling of unknown objects

The integrated IDS industrial cameras ensure a reliable, precise image capture that is needed for Fizyr's software algorithms which provide over 100 grasp poses each second, including the classification to handle objects differently. The software also performs quality controls and detects defects to prevent damaged items from being placed on a sorter – with the help of IDS cameras as sharp eyes of the automatic system.

Thus these algorithms are able to provide all relevant information about segmentation as well as about classification of type of parcel (including box, bag, envelope/flat, tube, cylinder, deformable, etc.). The system recognizes outliers or non-conveyables (i.e. damaged goods), best possible grasp poses in 6 DoF (degrees of freedom) and multiple ordered poses per object. It allows sensors or robots to deal with closely stacked or overlapping objects, highly reflective items and apparel in polybags, white-on-white and black-on-black flats, as well as transparent objects.

The system robustly segments unknown parcels, even under harsh lighting conditions.

uEye CP
Data acquisition
Data acquisition

Their extensive features qualify both Ensenso 3D camera models for the wide range of demanding applications in the supply chain and logistics industry, such as 3D object recognition, classification and localization (e.g. quality assurance, commissioning), logistics automation (e.g. (de-)palletizing), robot applications (e.g. bin picking), capture of objects up to 8 m³, (e.g. pallets) or automatic storage systems.

Fizyr has integrated the Ensenso SDK in its software using a modern and fast wrapper. A great advantage is the 2D/3D combination, which allows the image from the 2D camera to be placed over the 3D point cloud as an overlay.

— Herbert ten Have, CEO at Fizyr —

This provides a more vivid impression of the scene. On the other hand, the camera image can be optimally adjusted to the robot coordinate system by means of "hand-eye calibration" in order to ensure a target-oriented gripping.

Outlook

The strong growth of e-commerce has increased the demand for larger packages. At the same time, the constraints imposed by the global pandemic have led both to a reduction in the available labour force and to an urgent need for physical distance in companies. To meet these challenges, companies are increasingly turning to robotic automation, in which artificial intelligence and machine vision play a crucial role. At the same time, this reduces costs and increases occupational safety.

The demand for high-quality modules for state-of-the-art automated picking cells is growing worldwide. System integrators' requirements for software and hardware components are increasing in equal measure. The Fizyr-IDS solution offers the following advantages:

  • As a plug-and-play solution, it enables easy integration into any system.
  • It is hardware-agnostic and thus enables customers to flexibly adapt their order picking system to their needs at any time.
  • It includes a robust, precise, flexible and easy-to-use 3D vision system that provides optimal object information at all times.

All of them are indispensable components of a future-proof automated "Good-To-Person" system.

The Ensenso N series from IDS.
The Ensenso X series from IDS.
Fizyr