Computer software that can practice with doing a task and improve itself along the way, is something we mostly know from sci-fi films. However, Radboud researchers have developed such software, that taught itself to create an image of a face from just a pencil drawing.

The research group developed a deep neural network, an artificial network of nerve cells, that is able to translate line drawings to a color photograph image. The software improved itself along the way. During the training stage, the researchers offered the network a large number of examples of line drawings and the original photos that matched them.

It goes too far to say that the software is thinking for itself, says Marcel van Gerwen, one of the two head researchers of the project. ‘More specifically, the software starts to recognise patterns. It learns about the stable features in its surroundings.’ Humans do that too, but Van Gerwen says that there is a big difference: ‘Humans do not get thousands of examples handed to them.’

The research group is rooted in several research fields. Van Gerwen’s cowriter Rob van Lier and PhD student Yagmur Güçlütürk are interested in perception, for example the way people look at art. Van Gerwen and PhD student Umut Güçlü use the data on a more fundamental level, for example to find out how neural networks work and what their relationship to the human brain is. One interesting thing the software teaches us, says Van Gerwen, is the relationship between facial features and the colour of skin. Based on just the line drawings, the software knows the color of somebody’s skin.

Right now, the group founded a spin-off company, through which it wants to develop the software for smartphones. ‘So much is possible. You could draw something on your tablet and convert it into the style of a certain artist, for example’, says Van Gerwen.