Imitating the brain: all you need to know about artificial neural networks

There are many AI technologies and methods of simulating (or maybe rather creating) intelligence (some are listed below). Artificial neural network (ANN) is one example of such (quite amazing) technologies. But what exactly is ANN and why is it so amazing? Check out our handy guide and introduction to ANNs and their practical applications. We hope you’ll have an enjoyable read!

What is an artificial neural network?

Let’s start with the basics. ANN is a computing system inspired by the biological neural networks that constitute human brains (in other words a set of simple computational units that process data, communicate with each other and work in parallel). The name “artificial neural networks” sometimes also refers to the discipline that deals with the construction and study of such networks. Simple networks have long been used in fields such as handwriting recognition.

An ANN is based on a collection of connected units, artificial neurons, which loosely model biological neurons which look like this:

Of course, the working principle of an artificial neuron is slightly different – the artificial neuron is a simple computational unit  (mathematical function)  that processes signals, communicates and cooperates with other similar units. The first artificial neuron was produced in 1943 by the neurophysiologist Warren McCulloch and the logician Walter Pits. The simplest neural network, consisting of one or several independent McCulloch-Pitts neurons (model of biological neurons), is called a perceptron.

Why use neural networks?

The exact workings of the human brain are still a mystery. Yet, some aspects of this amazing processor are known and ANN mimic a number of these surprising features of the brain:

• high degree of parallel transmission pulses
• ability to deal with incomplete, inconsistent or “noisy” information
• small size and low energy consumption
• flexibility of action – learning by adapting to changing conditions without programing actions
• high resistance to damage and death of some cells

These features of ANN give a number of advantages:

• ability to learn from examples, just as a person learns from experience and repetition
• ability to generalize (classify) phenomena, e.g. division of transformer damage into types
• ability to interpret dependencies and phenomena containing incomplete/erroneous information, e.g. in a curve approximation problem with given energy characteristics
• parallel information processing – particularly relevant for real-time processing of information, e.g. a Mitsubishi Neurocomputer used for speech recognition and image processing
• low construction cost when compared to information processing speed – a large number of simple (and cheap) processors
• resistance to system damage – thanks to a large number of neurons, ANNs will not necessarily work erroneously once some of the neurons are lost

Let’s also look at a comparison between a classical computer and the “neurocomputer”, which is understood as the next step in ANN development – a computer imitating the biological neural network and emulating essential neural components:

Algorithm storage

Learning time

Processing

Arithmetic calculations

Heuristic capabilities

Image recognition

Dataset searching

Compression/decompression

Different types of artificial neural networks

A common feature of neural networks is that their structure consists of neurons connected by synapses. The synapses are associated with weights, i.e. a numerical value which interpretation depends on the model.

Most likely, it won’t be a surprise that there are multiple types of neural networks and that ANNs may differ, for example in terms of topology (architecture) or learning methods.

Architectures of neural networks:

Recurrent neural network (RNN) – a feedback network where connections between nodes form a directed graph along a sequence. RNNs can use their internal state (memory) to process sequences of inputs. Examples of this type of networks include: ART and BAM networks, Hopfield networks or the Hopfield network modification called a “Boltzmann machine”.

Feedforward network – allow signals to travel one way only. A special case of this type of networks are multilayer networks, which are the most popular ANN architecture.

Cellular neural network (CNN) – in this type of network, communication is allowed between neighboring units only. Typical applications of CNN include real-time processing (especially images).

How exactly networks learn?

The possibility of learning has most likely attracted the most interest in neural networks. ANNs learning process can be categorized into two general paradigms:

Supervised learning (“with an external teacher”) – this approach uses learning sets, input and output data. During learning process, the network adjusts the weights in order to limit the differences between the expected and learned value.

Here are some examples of supervised learning methods:

• Delta method
• backpropagation (BP)
• fast propagation method (BP modification)
• LM method
• conjugate gradient method.

Unsupervised learning (“without an external teacher”) – based upon only local information, network learns by modifying its parameters through an analysis of reactions to previously unknown stimuli. As a consequence of learning the network makes a division into classes/categories.

Below are examples of unsupervised learning rules:

• Hebb’s rule
• Winner-takes-all rule (WTA).

What ANNs can do?

ANNs have many practical applications, and many new ones are likely to emerge in the near future (so maybe we should ask what neural networks cannot do). In general, neural networks are best at identifying patterns or trends in data.

ANN applications can be divided as follows:

• data association (classification) – assigning an event to a specific class
• data analysis (group detection) – for example allowing for signal filtering
• feature extraction – allowing for reduction of data size (compression)
• approximation – ANN learns the function shape based on the knowledge of points
• prediction
• optimization (including combinatorial problems)
• control systems – ANN can automatically interpret the situation and respond to it on the basis of patterns of reactions to specific stimuli
• image recognition and reproduction – recognizing an image and assigning it to a given group, even if the distortion is quite high
• memory – the possibility of selective downloading in a very short time.

Neural networks have broad applicability to real world business and technical problems. Let us have a look at several examples from a domain which is really close to us – energy industry:

• optimization of tap-changer setpoints in distribution networks
• real-time optimization of capacitor bank setpoints in distribution networks
• forecasting of the peak power demand and the electricity consumption for selected groups of consumers
• analysis of the influence of non-generation factors on the quality of electricity consumption forecasts
• support for short-circuit analysis in power networks
• forecasting power generation from RES (4RES Project)
• diagnostics of power equipment, including transformers, turbine sets, nuclear power plant systems, leakage diagnostics, damage forecasting in a wind turbine
• system security assessment (e.g. assessments based on breaches, lack of breach of limitations, but also the occurrence of load coverage or the lack of it).

ANN and forecasting generation from renewable energy sources

To conclude we would like to present one example of ANN application in our research and development activities: the 4RES project.

It’s very important for RES operators to forecast the energy production in various time frames and with different temporal resolutions. To answer these challenges our team developed 4RES system providing means to forecast the production from RES by using artificial neural networks (layered networks, to be precise). The system was implemented at one of the trade operators where its effectiveness was assessed and confirmed – forecasts error rate proved to be satisfactory by reaching only about 10%.

4RES – forecasting generation from RES

You might be also interested in…