Small Automation, Big Impacts: Precision Object Detection with 3D Machine Vision

7 min read

In manufacturing, production tracking is used to measure, analyze, and improve visibility throughout the manufacturing process, from raw material to completed product.

Finished goods inventory is what manufacturers depend on to generate revenue. An accurate count of sellable goods is critical to maintaining a competitive edge. Manufacturers are constantly looking for the best tools to simplify their processes and improve efficiency.

Image analysis (2D machine vision) has been used in manufacturing to automate the counting process. However, it has its limitations:

• Contrast dependant: target objects must have enough contrast against the background (typically the conveyor belt). Such a system will have difficulties detecting dark objects on a black conveyor belt or light-colored objects on a white conveyor belt.
• Lighting disruptions: 2D machine vision relies on constant lighting (intensity) to capture images for image analysis software to process. Shadows cast by workers nearby or changes in ambient lighting have been known to affect outcomes.
• Overlapping objects: In many applications, items can be placed next to or on top of each other. Without height information, 2D cameras can have difficulty discerning individual objects.
• Size variance: It is harder to accomplish object detection when target objects vary in size. Large objects are also easier to identify than small ones as they only take up a few pixels within a scene. Increasing the camera resolution may help but will require more processing time and computing power.

The following two automation journeys illustrate how 3D machine vision using laser triangulation improves a factory’s production capabilities.

Rebar counting in steel manufacturing

Steel is one of the most important construction materials in the world. As high as 52% of the steel produced worldwide in 2019 was used by buildings and infrastructure projects. One of the essential products is rebar, which is used to reinforce concrete and steel structures.

Steel manufacturers must sort and bundle the rebar before delivery. The number of rebar in each bundle can differ from order to order, depending on the product grades. Most manufacturers still use manual and mechanical counting methods, which are both slow and error-prone. Some manufacturers are known to err on the side of caution by packing extra rebar, further adding costs to production. Accurately counting the rebar quantities is a mission-critical task that directly relates to the company’s bottom line and customer satisfaction.

In 2019, a study published by IEEE (Institute of Electrical and Electronics Engineers), Fernández et al. pointed out why 2D image analysis has been found to be unreliable for this application:

• Rebars do not have enough contrast from the background (conveyor belt)
• Rebars with diameters smaller than 11 mm (7/16 in) are difficult to discern
• The system cannot accurately detect rebars when they arrive on the conveyor belt attached

How 3D machine vision helps count rebars accurately

3D scanning by laser triangulation is accomplished by the scanner’s imager (camera) seeing the laser light reflected off the target object – the contrast between the object and the conveyor belt is inconsequential to the scanning process. In addition, laser triangulation does not require additional lighting as the laser itself is a source of illumination, making 3D machine vision favorable in this industrial environment.

While multiple ways to capture 3D point clouds exist, laser triangulation is designed to scan fast-moving scenes, such as a conveyor belt. A 3D scanner using laser triangulation can capture up to 1,250 scans per second (800,000 points per second). Depending on the application requirements, the point resolution can be down to fractions of millimeters. Rebar diameters of 11 mm or smaller can be represented in point clouds quickly and accurately.

Accurate 3D representation enables precision object detection

System integrators can easily develop analysis algorithms to process the 3D profiles captured by 3D machine vision scanners. Point clouds are a series of three-dimensional coordinates that correspond to the surface being seen by the laser beam. Mathematical relations can be derived to find point clouds of a conjoined rod, which may only appear partially on the camera. The high-resolution point cloud data is an accurate object representation that helps the counting system identify individual rebar regardless of length or position. The experiment performed by Fernández et al. concluded that counting 8 mm (5/16 in) rebars with laser triangulation improves the accuracy by almost 330%. The improvement also applies to counting larger rebars, with the system achieving a 0% error rate on the 12 mm (approximately 1/2 in) rebars.

Small Automation, Big Impacts is a collection of industrial automation journeys made possible by 3D machine vision. 

We believe sharing these stories is a way to stay connected with our partners and positively impact the manufacturing community.

This free eBook also includes a bonus chapter to help automation professionals easily decide when to use 2D and 3D machine vision. 

All this is just one click away in our new eBook, Small Automation, Big Impacts.

Download eBook

Monitoring production yield with burrito counts  

Driven by the rising cost of raw materials, yield monitoring and analysis are critical tasks in modern food plants. A food maker’s production yield indicates how efficient and optimized its processes are. Real-time process monitoring and data collection can help food manufacturers establish attainable production benchmarks and minimize material wastage.

A national frozen burrito factory in California is looking for ways to count finished products using faster machine vision technologies. Initially using a 2D counting system, the plant quickly realized the system was not fast enough for its conveyor belt. The conveyor belt could only go as fast as the counting system could process the images. Similar to the rebar counting case, the system had difficulties discerning two conjoined burritos, making the count an imprecise process.

Never miss a burrito with 3D machine vision

To accomplish the burrito counting task, Hermary’s SL-1880 scanner is set up 1.3 m (51 in) away from the conveyor belt. The height is chosen so that the laser’s region of interest covers the conveyor’s full width, which means only one scanner needs to be deployed per production line.

The 3D scanner creates profile scans from the surface of the burrito. With a burrito’s average length and width being 15 cm x 4 cm (6 in x 1 ½ in), each burrito is represented by approximately 2,300 data points using Hermary’s SL-1880. The data is sent to a PC that processes and parses the points into the geometric shape of a burrito. Using the burrito’s surface area as a unique identifier, the system can easily recognize each incidence of a burrito, including conjoined ones.

With 3D machine vision scanners, the counting system improved its accuracy rate to 99.9%. Additionally, unaffected by background contrast, the system can scan correctly even when the conveyor belt is light-colored or covered with flour. The system scans and counts up to 35 burritos per second, or 2,100 burritos per minute, allowing the production line to travel full speed at 3 m/min (10 ft/min).

The journey continues

Laser-triangulation-based 3D machine vision is fast and produces deterministic point cloud data that allows software programs to detect a target object accurately and consistently. 3D machine vision is notably powerful when scanning free-formed objects or objects with color variations, such as crops or natural resources.

Vision Guided Robots

Object detection is also a fundamental step in working with vision-guided robots (VGR), which has the added advantage of production flexibility versus robots that move along pre-programmed paths (blind pick-and-place robots, as commonly used in automotive manufacturing).

VGRs not only help relieve the labor crunch but also helps manufacturers – especially small and medium-sized – to complete customized orders without much human intervention. By feeding robots with a 3D scanner’s data, a VGR can move and perform tasks in relation to the object coming down the conveyor belt. Each object may be located differently on the production line or assigned a different task. The VGR uses the 3D machine vision data to move into position and complete the required task (i.e., baguette scoring, cake icing decorating.)

Commercial bakeries have been using Hermary’s 3D scanners to automate delicate tasks traditionally only associated with artisanal bread or made-to-order cakes. While the potential for VGRs in sectors like agriculture or commercial bakeries remains largely untapped, more and more companies are capitalizing on the potential of 3D machine vision to get ahead of the competition.

Big impacts of small automation

Industry: Steelmaking and food & beverage

Safety

• Removes humans from working near hot steel bars
• Reduces human coming into contact with food products 
• Empowers the control room with data for predictive maintenance

Operation

• Accurate 3D representation of target objects
• Fast scanning speeds
• Background contrast independent
• Extreme immunity to changes in ambient lighting
• High-resolution 3D data down to fractions of millimeters

Future expansibility

• Fast scanning speed to match faster conveyor speeds
• Upskilling workers to perform more value-added tasks
• Vision-guided robots for customized orders
• Redeployment to handle other free-formed products

Bottom line

• Accurate product count and inventory level
• Improved customer satisfaction
• Reduced reliance on manual work

Next: Read about how 3D machine vision helped industrial bakeries gain an artisanal touch