Deep learning: What exactly is it?

Deep learning enables breakthroughs in all fields of artificial intelligence. Whether it's image recognition, understanding language, machine translation or autonomous driving. But what is actually behind it all? And when does it make sense to use deep learning?

Deep learning is a special method of information processing and a sub-area of machine learning that uses neural networks and large amounts of data to make decisions.

The learning methods are based on the way the human brain works, which also consists of interconnected neurons.

This design makes it possible to repeatedly link and expand what has been learned with new content on the basis of new information.

As a result, the software is able to make predictions or decisions.

Based onhttps://www.bigdata-insider.de/was-ist-deep-learning-a-603129/

History of deep learning

Deep learning is based onneural networks. They are not new. The first ideas and research were already being conducted in the early 1940s. The idea is to recreate the structure of the brain from interconnected neurons. While I was still studying in the 1990s, neural networks were more for freaks and hardly found their way into practical applications. In particular, the performance of the available computers very quickly reached its limits and the available data volumes were not nearly large enough.

However, this has changed rapidly in recent years. The introduction of high-performance hardware for matrix operations from the graphics sector has made it possible to set up and train even more complex network topologies. There are now even processors specially optimized for training neural networks, such as Google'sTensor Processing Units.

At the latest with thevictory of Google's Alpha Go over one of the world's best Go players, an enormous hype developed around deep learning in all areas of artificial learning. This even extends to creative applications such as generating images or composing pieces of music.

What is deep learning?

Neural networks are a structure of interconnected neurons, weights and thresholds. The topology, i.e. the arrangement and networking of these neurons with each other, plays a major role. Deep learning topologies consist of many layers of neurons, each of which typically abstracts or responds to certain features, i.e. properties of the input signal. This deep nesting is particularly powerful and is responsible for the name "deep learning". These are oftenconvolutional networks, which are very well suited for image recognition, among other things.

There are alsorecurrent networks that have a temporal memory and take into account the history of a signal. This is particularly important for applications in the field of language or for time series analyses, where references to previous input values play a major role.

And how does deep learning understand me?

In general, neural networks can only process vectors or matrices. The mapping is relatively trivial for measured values or images. An input neuron can be assigned to each sensor or pixel. For speech, the mapping is more complex: if you were to simply assign an input neuron to each word, the calculation would be very inefficient.Word embeddings were therefore introduced , which place words in a multidimensional vector space.

The matrix resulting for a text from the transformation can be used as an efficient input matrix for the network. Interestingly, as a by-product of word embedding, semantics are also created within this vector space. Ideally, it can even be used for calculation and prediction. For example, if you know the vector that connects Spain and Madrid, you can also determine the capital of France using vector algebra.

Learning, learning, learning...

In order to be able to use a neural network for the classification of images or texts, it must first be trained. For this purpose, a signal - e.g. a picture of a horse - is applied to the input neurons. The weightings and threshold values within the network are then optimized until the correct output neurons are activated for thehorse content.

This is an extremely computationally intensive optimization process that requires a very large amount of training data. Those who already have access to a large, quality-assured reference set, such as professionally classified patents or image databases with keywords, are in luck. Or those who can collect data on users and their behavior on a large scale, such as the major social media platforms.

The application of the calculated network, e.g. for recognizing image content, is relatively simple. It can also be easily implemented on mobile devices. This asymmetry is partly responsible for the breakthrough of deep learning, as once a network has been calculated, it can be used very cost-effectively in embedded computers for electronic devices, control systems or vehicles.

Another catalyst for the breakthrough of deep learning is the democratization of machine learning. Very powerful frameworks for implementing neural networks are now freely available, meaning that the wheel does not have to be reinvented again and again. A large community can therefore focus entirely on the further development of topologies, training methods and models, which leads to extremely rapid progress in various areas.

Deep learning as a universal solution?

In general, the great advantage of deep learning over other methods is also its Achilles' heel: the performance of a network scales with the amount of available training data. This means that deep learning can only be used sensibly with significantly large, tapped data volumes. Then, provided the necessary computing power is available, it is also really fun. And nobody has to worry about the tedious creation of rules or in-depth data analysis to determine relevant parameters or properties.

Theblack box aspect of deep learning should not go unmentioned. Just as we still do not understand the processes in the human brain, we cannot say why a certain decision is made in deep learning. In particular, scientific studies have shown thatnetworks for image recognition can sometimes be easily fooled with specially prepared images. For example, if certain patterns or noise are applied to an image, traffic signs can suddenly no longer be recognized.

In practice, we at Empolis are working, for example, on the application of deep learning to analyze textual medical findings and for patent classification. We are currently pursuing a hybrid approach that takes into account both images from patents and the textual description of the patent.

What is deep learning best suited to?

Deep learning is a very useful addition to our big data analysis toolbox for problems for which large amounts of data are available and deterministic traceability is not absolutely necessary. For all other scenarios, classic, statistical, deterministic and rule-based approaches continue to make sense.