An introduction to AI in sorting technologies

In general terms, artificial intelligence (AI) is intelligence demonstrated by machines. This definition indicates that AI covers a range of topics. For instance, a rather simple algorithm implemented via electronic circuits could be considered AI as soon as the system began to show intelligent behavior (e.g., if the temperature of the machine reached a certain threshold, the machine would know to shut off on its own).

Improvements in AI have enabled the technology to perform deep learning, image recognition and speech recognition. Although AI research has been around since the 1940s, advancements in computational power during the past few decades have finally led AI to achieve more remarkable milestones. In its earlier versions, AI beat the best human chess player in 1997. Then, in 2017, AI was able to beat the best human Go player.

AI has been used for years in recycling applications, as well. Regarding AI’s use in machine vision, how exactly do machines identify an object and whether it’s something that needs to be kept in the stream or removed? These are key challenges to solve related to the use of AI in sorting technologies for recyclables.

History of AI in sorting

AI’s use in sorting started with systems in the 1970s to 1980s. These systems were based on optical sensors and electronics that compared gray values or colors. Based on the ratio between these colors, the electronic circuit would make a rules-based decision whether to keep or eject the material. For instance, the first reverse vending machine (RVM) recognized the shape of a bottle based on the shadow it generated, which was detected by prepositioned optical sensors.

In the early 1990s, pixel-based classification of gray-scale and color camera images was used in combination with custom-made electronics, which limited the capabilities of the AI in terms of thresholds and decisions. With the emergence of personal computers (PCs), it became possible to use this technology for the classification of images.

Customized camera technology was used to acquire specific spectral properties and better clustering possibilities, leading to improved AI accuracy. This made it possible to assign each pixel with a specific class of material based on its spectral content. Color was no longer the only identifying criteria.

This technology was then combined with object recognition in the late 1990s, which made it possible to cluster different pixels of similar properties and combine them into an object.

By the 2000s, hyperspectral imaging systems became available, and the power of PCs further increased. Artificial neural networks (ANN) started to become available for classification problems in data processing. Based on previously trained samples for the specific application and machine, this class of AI could now combine different features and properties to make one classification. As a result, more complex materials could be detected and another level of sorting accuracy was achieved.

Later in the 2000s, so-called support vector machines (SVMs) became available. Although it sounds like a physical machine, these are mathematical models that allow a machine to define clusters in multidimensional space. Storing the results in tables on the physical sorter improved the performance again.

Common to all previously mentioned forms of AI used for sorting is the fact that the so-called training or learning aspects of AI must be supervised. In the very simple example of the RVM from the 1970s, the engineer had to physically place the optical sensor in the correct location, and a set of labeled samples needed to be available for teaching the system before putting the sorter into operation.

AI today

Today, the initial teaching of the system requires a computer vision engineer to define the relevant features for the sorting task. This generates feature vectors from the image data, which are then used in conjunction with the labels for automatically training the ANN or SVM. Because the training is done automatically without interaction by the engineer, this approach is called unsupervised learning.

The next step in the evolution of AI in sorting is deploying deep learning methodologies that became available in the 2010s and are now used in a range of applications. These types of networks were invented decades ago. Because of a massive increase in processing power in modern graphics processing units and millions of generally available and labeled images, it is now possible to apply them to practical problems.

So-called deep convolutional neural networks are still an ANN; but, compared with the early derivations, they have a much larger number of layers and neurons. Consequently, the networks are more powerful. However, they also require a lot more training data than traditional approaches.

The major advantage of convolutional neural networks is that the feature extraction step also is performed automatically during the training of the network. As a result, a computer vision engineer is no longer required to manually define the features relevant for the task. Typically, the first layers of the network generate features, which are integrated into more complex features in the following layers and then classified in the last layers.

These networks can be combined almost like building blocks, with each one being pretrained for a certain task. By doing this, the design can be adapted to the application at hand. Deep learning technologies have a major impact in image recognition in particular.

A glimpse into the future

The current phase of AI development—especially the deep learning aspects—will enable the recycling industry to tackle currently unresolved challenges.

Today, a handpicking station at the end of a line is still needed to improve the quality of the final product to the desired level. An example could be seen with silicon cartridges, which are not desirable in a polyethylene stream. To pick them up with a robot, or to eject them through a last optical sorter, they would need to be detected first.

For this capability, AI and deep learning will play an important role in improving efficiency. Combining these new forms of AI with the potential of big data (e.g., with the data we already can collect from the machines today) will open even more opportunities to increase production, reduce cost and improve quality.

There’s this idea that today’s AI is like finding a free lunch and an ugly duckling. Both ideas are actual mathematical theorems that relate to the topic of artificial intelligence.

The first theorem basically states that no single AI solution is superior to all the others for a specific application. Each solution can have certain benefits that come at the cost of some disadvantages elsewhere—hence, there is no such thing as a “free lunch.”

The ugly duckling theorem is similar, stating that no optimal feature set exists for all applications. Even if we could find generic AI that solves many different challenges, it would not fit at least one application or problem and would not provide a suitable solution—making it the “ugly duckling.”

With this in mind, we should stay modest in our expectations regarding what is possible with convolutional neural networks and deep learning. Plenty of examples are available where deep learning is solving difficult, loosely structured recognition problems, but with other sorting tasks other AI approaches will have better performance. Finding the right combination of different types of AI was key in the past and will remain a key to ensuring the best sorting performance for recyclables in the future.

Artificial intelligence has been deployed in the recycling industry for quite some time. Yet, the possibilities that deep learning can offer when the fields of machine vision and machine learning are brought under one umbrella are new and exciting for the industry.

The authors are vice presidents with research responsibilities for Germany-based TOMRA Sorting GmbH, www.tomra.com/en/sorting/recycling, part of Norway-based TOMRA Systems ASA.