Neural Networks - Basics

What are neural networks?

Neural networks are algorithms that get input data and adapt parameters on an internal (unvisible) model so it works well on the known data and (hopefully) delivers good results on new data.
Neural networks are not new, the first approaches have been introduced already in 1943 by Warren McCulloch and Walter Pitts under the name of "Threshold Logic" models. In the '80s the effective backpropagation algorithm was introduced which spread the usage of neuronal networks. But still large scaled applications have not been used as they required huge datasets and storage. This is why they got popular in the last 10 years, when those parameters do not provide any difficulties anymore.

How do neural networks look like?

If you see pictures of neural networks, they will usually look like this:

The round forms are called nodes, they are ordened in different colums called layers.
- the first layer (on the left) is called the input layer. It determines the number of input parameters.
- the last layer (on the right) is called the output layer which holds the calculated results of the algorithm.
- the columns in the middle are called hidden layers (unvisible from outside).

For a neural network you have the input x (here x is a vector with two components) that are passed to the input layer, mapped through the network and create an output (here a vector with two components), that in the ideal case is a good approximation to the known output y. During the way through the network, every input component is multiplied with parameters and enriched with some constant bias, the composition is then passed to a node, in which the activation is applied.
From the error of the prediction with the choosen parameters the algorithm can calculate deltas and adapt each parameter, so that in the next run the prediction will get closer to the actual known result.
So the algorithm learns by minimizing the error.

An example for supervised learning is the recognition of handwriting: by feeding the system with lots of letter sets of different hand writing styles and providing the correct result the system can find substructures and compare them to a new dataset. Then it can calculate probabilities that the input data matches one of the known letters and propose a solution. This often cited example is used as introducting program in Google's Machine Learning Platform Tensorflow, which is a very powerful tool for the interested community.

Machine Learning

What does "machine learning" mean, how can a computer "learn"?

The term "learning" is used in this context to refer to the fact that a machine uses new data in order to better up previous results. For sure the machine will not develop a brain or a physically similar organ. But for optimization or minimizing tasks, especially tasks including complex calculations and transformations, a machine controlled adaption is a useful and often indispensable tool. The basics for such tasks are often machine learning algorithms like artificial neural networks or clustering algorithms which run iterative trying to minimize errors in each calculation step.

Machine learning algorithms usually can be devided in two main groups:

• Supervised learning algorithms, in which the output is known and the rules are being trained by using the input and trying to minimize the error between the predicted output and the given result. Neuronal networks are here the most promising examples, they are called like that as they reflect the way our brains work: given some input the human brain learns by trying out and correct until it finds the perfect rule to explain the result.

• Unsupervised learning algorithms in which data is given to an algorithm which then tries to find pattern in the data. As an example think about astronomical data: if you can cluster the stellar data you could find a structure in it and learn about the past and future.

In addition modern algorithms can also be active learning algorithms, in which the input can be completed by additional requests for input trying to minimize the numbers of these additional inputs.

What are these algorithms needed for and why are they considered promising?

Machines have the capacity to calculate fast and storage is cheap nowadays, also in presicion they are unbeatable and often parallelize their tasks. As nearly unlimited data is available and often more data leads to better results (not always, an intelligent way to sort out data is one of the reasons for a data scientist!) machines and computer visualizations are essential reasons for machine supported analysis. In addition cloud computing and distributed systems improve the way data is collected, loaded and analyzed.
Internet of Things scenarios are considered the modern way to improve business processes and drive Industry 4.0 by collecting huge amounts of (sensor) data. To analyze those datasets machines are not only helpful, but necessary.