Polarization in real time
Use of on-camera data pre-processing of IDS polarization cameras
With the knowledge of polarized light, reflections and highlights can be reduced or contrasts of fine structures increased in industrial image processing. Even physical properties such as material tensions that act below the surface can be visualized. To determine the polarization information, up to now 4 sequential images with different orientation of the polarization filter and subsequent PC processing were necessary. With Sony's Polarsens technology, a single recording is all that is needed. Additional data pre-processing converts the raw data in real time from IDS polarization cameras into useful image formats, for more effective further processing or evaluation in the host PC.
What is the difference between an IDS polarization camera and a normal camera? What are the possibilities of pre-processing the polarization information in the camera? In order to understand the application of these special cameras, a short technical excursion is necessary.
We already know that light is electromagnetic waves, which contain all the important information to describe the characteristics of light that are visible to us. Both the human eye and the image sensors of cameras can capture the direction, intensity (amplitude) and color (wavelength) of light and digitize it for image processing. The wavelength range of light is only a small part of the entire spectrum of electromagnetic waves.
Polarized light
Another feature that has so far mainly been used in image processing with special illumination, filter glasses and optics is the polarization of the waves. Polarization does not describe the direction of the light beams themselves, but the direction of the wave amplitudes in which the light particles (photons) oscillate perpendicular to the direction of propagation. The waves of unpolarized light oscillate in different directions. Incandescent lamps or sunlight are examples. If the photons move in only one direction, we speak of linearly polarized light. Special polarizers absorb or attenuate incident light that oscillates in all but one plane - its polarization axis - resulting in an almost linear polarization.
The behaviour of linear polarization filters
The influence on the resulting intensity of a light wave with the initial intensity I0 when passing through a linear polarization filter was already proven in 1808 by the French physicist Étienne Louis Malus and mathematically described with the "Law of Malus": I = I0 . cos²(Φ)
Accordingly, light waves that are twisted in relation to the optical axis of the polarizer are not generally blocked, but their intensity is attenuated depending on the angle and polarized in the direction of the filter. The behaviour of linear polarizers is often misleadingly described or represented. Unfortunately, the graphic above is another example of this.
What do reflections have to do with polarization?
On highly reflective surfaces, such as glass, light rays can be deflected. Depending on the viewing angle of the object, unwanted reflections in the direction of the observer may then occur. The following example with a car shows how car windows become opaque due to such light reflections. Objects or persons in the interior are thus hardly recognizable - become almost "invisible".
If light reflections occur on non-metallic surfaces (as here on glass), physical processes cause a natural polarization of the light. This means that this special polarized light component and thus the glare effect can be reduced by using an appropriate polarizing filter. The unpolarized part of the light can still pass through the filter.
Light reflections on a windshield can be removed by polarizing filters
Reflections on metallic surfaces cannot be eliminated by a polarizing filter alone! On metallic surfaces the light is not polarized, but only reflected. Therefore, despite the polarizing camera, it is absolutely necessary to mount an additional polarizing filter on the illumination, depending on the application, in order to suppress disturbing light reflections.
The use of already polarized light (e.g. through a polarizing filter foil) further reduces the generation of reflections on surfaces. Special computer displays (without touch function) are therefore provided with an "anti-reflective" film. Although the light intensity decreases with narrow viewing angles, no reflections are created. If you look at these displays almost parallel, the screen seems to be dark!
Importance of polarization for image processing
For the pure perception or digitalization of light it makes no technical difference whether we are dealing with polarized or non-polarized light. In digital image processing, it is primarily the illumination of a scene that is essential to make certain features visible. But everyone who has ever taken a picture with lighting knows the advantages and disadvantages. The unwanted shadows are rather easy to control with additional or stronger lighting from different directions. In combination with diffuser screens, a more homogeneous illumination of the scene can be achieved. But in connection with light, one always has to fight with disturbing reflections and fading, which often leaves important features undiscovered. If a mirroring erases information, for example, because a car driver's face is outshined by the reflection on the windshield, this cannot be restored after image acquisition with any image processing technique. Especially in industrial image processing, diffuse scattered light is often used to reduce the disturbing influences as much as possible. Depending on the application, however, optimum lighting can be very expensive.
But not always more is better or more suitable! In the case of light, the essentials can also be made visible through targeted filtering of the disturbing influences. The knowledge about polarized light, how it is generated and how to influence it, provides approaches to solutions. Polarization filters are used in image processing to reduce reflections or highlights but also to increase contrast. They are also useful in stress analysis to visualize physical properties below the surface.
Advantage polarization
Since light is polarized not only by superficial but also by other physical properties such as mechanical stress or light refraction, object features and defects can be made visible that cannot be detected by normal image sensors.
Making the invisible visible
Material defects or object features that are sought after do not always become visible through more light or by eliminating disturbing influences. In the following picture example it only seems as if the contents that are important for us are exposed more or their light intensity is increased. That's not so. The light is polarized by the reflections of the light from the stains and scratches.
Using the degree of polarization, stains and scratches become visible during glass inspection
The fingerprints and scratches can only be seen very clearly here by visualizing the "degree of polarization", which makes the polarized light rays more clearly visible than unpolarized light. But normal camera sensors do not record the polarization of light. This means that the information is lost during the acquisition. A visualization or even a subsequent polarization correction by image processing cannot be applied anymore, as even in a RAW-file it is not recognizable anymore where light was captured with which angle and degree of polarization.
To calculate such an image it is necessary to capture the polarization of the light of this scene in numbers. This makes it possible to subsequently process an already digitized image or to make "invisible" features visible to us. This quantization of the polarization is very easy to achieve with Sony's Polarsens technology, for example. This means that with cameras such as our GV-5080CP-P (GigE Vision) or U3-3080CP-P (USB3 Vision), which include the Sony IMX250MZR mono Polarsens sensor, polarization information can be captured in a single image.
IDS camera with 4x linear polarizer
IDS polarization cameras with the Sony IMX250MZR mono polarization sensor are available as GigE Vision and USB3 Vision versions.
The technical data sheets
- U3-3080CP-P-GL Rev.2
- GV-5080CP-P-GL
can be downloaded from our website.
A single image is sufficient to capture the polarization information together with the image content. No special accessories such as a polarized light source or polarizing filters are required. This is made possible by the innovative design of the Sony sensor.
The "four-directional polarizer" located between photodiodes and micro lenses generates a sensor raw image with four polarization directions (0°, 45°, 90° or 135°) in one image by the principle of linear polarization filters. A different intensity is measured for each angle of the polarizing filters. Four adjacent pixels in a 2x2 cluster with their four different polarization filters form a "calculation unit". The real 5 megapixels of the sensor are thus divided into 4 smaller images for one polarization angle each, but their image content reflects the same moment. This means that the camera has the optimum output data for calculating the polarization information - and that with every shot.
The four single images are available with 1.26 MP in a reduced resolution and brightness, which can lead to strong noise of the result values during the following polarization determinations in the border area. Therefore, make sure that you have suitable and sufficient lighting when capturing images.
The mathematical calculation basis for the polarization state of each sensor calculation unit is the Stokes vector. With the help of its 4 components, the degree of polarization and angle of polarization can be determined from the four measured light intensities.
OnCamera polarization
Industrial cameras provide the image material for digital processing. Even though the raw format of an image sensor offers the most possibilities for subsequent image processing, it is not suitable for direct visual inspection, for example. Through preprocessing, important and often needed results could be calculated directly. It also saves time and PC computing load. In combination with Sony Polarsens technology, other useful image formats are therefore available in addition to the sensor raw format, which can provide optimal output data for image processing on the PC.
Starting with camera firmware version 2.4, uEye CP polarization cameras are capable of independently determining the direction and degree of polarized light from a single image acquisition using the raw image data of the "four-directional polarizer" by pixel preprocessing. This allows new image information to be calculated and visualized with the polarization data, for example to make a contrast visible. With predefined, selectable image components, the user can, for example, filter out disturbing light reflections directly from the sensor raw data or make reflective object features visible in almost complete darkness even before the image is transmitted to the PC - with just one click!
Advantage "oncamera data pre-processing"
- It's so easy - Simplifies the work of the image process engineer, who already receives the optimal data for further processing from the camera.
- Efficient - Saves PC resources. Camera and PC share the work effectively.
- Vision-compliant - Result data from the camera is available to any standard vision application without the need for additional or special software.
Application specific image formats
For the most important applications, 8 image formats are offered for selection, which are calculated in real-time in the camera's FPGA.
Raw
Image format: 8 bit mono, 2448 x 2048 pixels (full sensor resolution)
In raw mode, the camera delivers the sensor data without preprocessing in full 5 MPixel resolution as an 8-bit grayscale image. With the intensity values of the differently polarized pixel filters, both angle and degree of polarization of the linearly polarized light can be determined. The raw data format is thus the basis for own PC-based evaluations and calculations in the customer application.
Intensity
Image format: 8 bit mono, 1216 x 1024 pixels (1/4 of the sensor resolution)
"Intensity" corresponds to the average value (m) of the four measured light intensities after passing through the "four-directional polarizer" with the four polarization directions 0°, 45°, 90° or 135°. It is calculated with m = (I_0° + I_45° + I_90° + I_135°) / 4
The image format represents the brightness without polarization information of light. It is therefore most comparable to the intensity value that a normal mono image sensor would capture.
IntensityNonPolarized
Image format: 8 bit mono, 1216 x 1024 pixels (1/4 of the sensor resolution)
This intensity value (I_min) only corresponds to the proportion of unpolarized light in the image. The polarized part of the light is calculated with I_min = m * (1 - DoP). With this image format, light reflections are already effectively reduced.
IntensityOnlyPolarized
Image format: 8 bit mono, 1216 x 1024 pixels (1/4 of the sensor resolution)
This grey-value image (I_amp) contains only the polarized part of the light. It is calculated using I_amp = I_max - I_min. Areas in which no polarized light is present become darker. The current brightness of the individual pixels is fully integrated into the image. Exposure changes affect the display. This distinguishes the mode from the polarization degree (DoP) mode, because here the absolute values and not the relative portions are output.
DegreeOfPolarization (DoP)
Image format: 8 bit mono, 1216 x 1024 pixels (1/4 of the sensor resolution)
The degree of polarization indicates the percentage of polarized light in relation to the total intensity. Due to the relative values, the DoP calculation is only slightly affected by exposure changes as long as no clipping occurs.
- The higher the degree of polarization, the better the angles determined are suitable for subsequent calculations and analyses.
- If the DoP values are too low, the polarization information becomes useless for analysis. There is a lot of noise in the image information.
In applications with polarized light DoP is an important value for the quality of the results or a good measure for the qualification of the right lighting conditions of a scene.
PolarizationAngle
Image format: 8 bit mono, 1216 x 1024 pixels (1/4 of the sensor resolution)
This image mode displays all calculated polarization angles (1 angle per "four-directional polarizer" with 2x2 pixels) as a gray scale image. The angles are scaled internally so that they are output in the range 0-180°. Negative output values therefore do not occur.
In areas with very little polarized light, the angular results vary greatly, since the angular information from the "four-directional polarizers" cannot be extracted cleanly by noise. This often means that a stable angle calculation is not possible.
When using the calculated polarization angles in an application, prior qualification by the degree of polarization (DoP) is recommended.
Combination of angle and polarization degree information
Using the HSV color space, the camera can produce a combination image format with more than one piece of information. By linking, for example, the angle and degree of polarization in an image, the results can be interpreted particularly well. This also results in a false-color display, which also visualizes the relationships.
Especially if the angle information cannot be extracted cleanly by noise, because very little polarized light is available, the results are mostly useless for meaningful further processing. In this case, a combination format provides a measure for validation in the same image in addition to the angle value!
IDSHeatMap
Image format: 24 bit RGB, 1216 x 1024 pixels (1/4 of the sensor resolution), HSV-based (H = angle, V = DoP, S = 100%)
In this combination format, the determined angle of polarization is combined with the degree of polarization. The brightness of the resulting image is scaled by the DoP. As a result, dark areas show that the angle information is not reliable because the polarization component is very low and the angle calculation can therefore be inaccurate. For bright areas, on the other hand, there is a very large polarization component and the angle can be reliably determined.
IDSColorMap
Image format: 24 bit RGB, 1216 x 1024 pixels (1/4 of the sensor resolution), HSV-based (H = angle, V = abs (mono - DoP), S = DoP)
In this combination format, the determined angle of polarization is also linked to the degree of polarization. However, here the actual gray value image is included in the calculation. The brightness here corresponds to the amount of the difference between the grayscale image and DoP. This makes the image brighter overall due to bright mono image areas as well as high DoP. Only areas where there is neither brightness information nor high DoP information become dark or black. Additionally DoP is used as saturation. Thus, strong colors indicate a high proportion of polarized light in the image area.
Configuration via the Vision Cockpit
The image formats of IDS polarization cameras can be set and used with any GenICam-compatible software. We explain the configuration of the camera parameters with the help of the Vision Cockpit, which you can install with our software development kit IDS peak. The image formats can be found as individually selectable image components in the ImageFormatControl node. If you have opened your camera with firmware 2.4, you can find the settings by entering for example "Component" in the search field.
You activate the individual image formats with the Vision Cockpit as follows:
- Disable image acquisition
- select the desired image format with the "Component Selector
- Activate image format with "Component Enable
- Restart image acquisition
The camera automatically switches to the required image format (e.g. "8 Bit Mono" or "24 Bit RGB").
Programming with IDS peak
To use the new image formats in your own application only a few lines of source code are necessary. The following source code blocks show the programming of the image formats with IDS peak in the programming language C#.
Retrieving all available image components
var imageComponentsNode = nodeMapRemoteDevice.FindNode<peak.core.nodes.EnumerationNode>("ComponentSelector");
var availableImageComponents = imageComponentsNode.Entries();
foreach (var entry in availableImageComponents)
{
display(entry.StringValue());
}Querying the currently active image component
var activeImageComponent = "";
foreach (var entry in availableImageComponents)
{
imageComponentsNode.SetCurrentEntry(entry);
if (nodeMapRemoteDevice.FindNode<peak.core.nodes.BooleanNode>("ComponentEnable").Value() == true)
{
activeImageComponent = entry.StringValue();
}
}
display(activeImageComponent);Select and activate an image component
imageComponentsNode.SetCurrentEntry("IDSHeatMap");
nodeMapRemoteDevice.FindNode<peak.core.nodes.BooleanNode>("ComponentEnable").SetValue(true);Summary
Polarization is a property of light that enables the recognition of object attributes that are invisible to the human eye and also "normal" image sensors. This makes it an important tool for digital image processing in applications with reflective or translucent surfaces. With the SONY IMX250MZR sensor and the oncamera pixel pre-processing, IDS polarization cameras are able to determine all necessary polarization information of the image scene with a single image acquisition and to provide this information in different pixel formats to the host PC for further processing or direct evaluation.
FPGA-accelerated algorithms make cameras more than suppliers of pure sensor data. In real time, they already provide meaningful evaluations that can be further used by any GenICam-compliant application via GigE or USB3 vision interface. IDS polarization cameras thus become part of the image processing and reduce the computing load of the host PC.
Test for yourself how easy it is to make object attributes visible with just one click before the image is transferred to the PC.