AI opens the door to new sorting possibilities

Deep learning uses artificial multilayered neural networks and a vast amount of labeled sensor data to learn how to classify or detect objects. For the detection and classification of images, convolutional neural networks (CNNs) primarily are used. The name is based on one of three types of layers used in this network architecture: convolutional layers, pooling layers and fully connected layers.

Convolutional layers connect neighboring pixels as inputs for the next layers, thereby detecting local structures in the image. Convolutional layers typically are stacked with pooling layers, which downsample the image and condense the information in the convolution layers. Image features are calculated by stacking these layers and are classified in the fully connected layers, the last of the three layer types. (See Figure 1 below.)

The ImageNET Large Scale Visual Recognition Challenge (ILSVRC) illustrates the performance of CNN architecture. ILSVRC evaluates algorithms for object detection and image classification on a large scale. In this challenge, deep- learning architecture defeated human performance in detecting and classifying 1,000 object categories.

The best classification and detection results are achieved using deep- learning techniques. Because of their extensive use and low cost, RGB (red, green and blue) color cameras were the first sensors evaluated. But an RGB camera cannot tackle all the sorting tasks in the recycling industry. Near-infrared (NIR) spectroscopy sensors offer a solution to this problem. This type of sensor in combination with classical machine-learning algorithms works fine, but its use with deep-learning algorithms could lead to even better sorting results.

New possibilities

The recycling industry presents a number of sorting tasks, and, for some of them, the results achieved by sensor- based sorting machines do not meet the required purity. These tasks are currently performed by humans and are subject to error. One example is removing silicon cartridges from a polyethylene (PE) stream. If these cartridges are not removed, the silicon will disturb the PE recycling process. This task is performed at a hand-picking station at the end of the sorting line.

The results from the ILSVRC lead to the assumption that deep-learning techniques could detect silicon cartridges with an accuracy close to or even higher than that of hand pickers. One might assume that the cylindrical shape of the silicon cartridges means the task is easily solved with a classical machine-learning approach. Yet, the problem is more complex than this approach can handle.

Variations in the cartridge opening, cartridges with or without tips, deformed cartridges or even partly demolished ones can affect sorting performance. All these objects must be detected and classified correctly.

After detecting and classifying the objects, they must be sorted as defined in the given task description. This can be realized in different ways, such as by using a robotic arm or a classical valve block. The latter approach can remove multiple identified cartridges efficiently and reliably at the high conveyor belt speeds used in industrial applications. The distance between the sensor measurement and the place of ejection is small and therefore the calculation time is limited to milliseconds. This might constitute a problem because the calculation performed in a deep-learning model can be quite complex. This is also the case during training, where the variables that form the network are determined by feeding thousands of input samples to the network, which requires massive storage volume and calculations. The time required for training can be reduced to a more feasible period of days or hours by parallelizing the calculations on graphic cards and deploying cloud technologies.

The network also could detect and classify other objects with nearly no overhead, allowing for better sorting according to the task defined by the operator. (See Figure 2 below). Deep learning used in combination with NIR spectroscopy can offer the recycling industry a new area of sensor-based sorting.

The future is self-optimization

New developments in the area of deep learning show that algorithms can perform tasks that were previously performed only by humans. The combination of different sensors observing the material stream and fusing their information via CNNs shows great potential to tackle the most difficult sorting tasks in the recycling industry.

By learning from example, future machines will adapt to changing material streams without being reprogrammed. The algorithms will be able to learn which objects have to be removed from the stream, even from only a few samples.

Sorting machines will be connected to the cloud and will contribute to a shared pool of information. False detections also could be incorporated into the learning process across multiple machines. Instead of delivering a sorter with specific software that is preconfigured to detect and eject a certain material, the capabilities, accuracy and efficiency of the connected sorter would improve over time.

Sharing status information allows machine data to be analyzed for diagnostic and maintenance purposes. Also, machines would be able to analyze themselves and could order replacements parts or services to minimize downtime.

These advances in the field of AI would solve many recycling sorting tasks that are not yet feasible to perform using technology. Sorting machines would be able to optimize themselves to achieve the best results for the given requirements set by the operator. That could mean that getting the most out of resources would become much easier.

Daniel Bender is team leader for deep learning, Dirk Balthasar is vice president and head of core R&D and Felix Flemming is vice president and head of digital Tomra Sorting for Germany-based Tomra Sorting GmbH, www.tomra.com/en/sorting/recycling, part of Norway-based TOMRA Systems ASA.