Automatic Design of Deep Neural Networks

BMBF-Project »Deep Topology Learning« (DeTol)

The development of the last years shows that machine learning and especially the subarea of Deep Learning will be an important component in the future, both in the scientific and industrial area. In the BMBF project »Deep Topology Learning« we are working together with other institutes on the acceleration of design algorithms.

From speech recognition and automatic image analysis to prototypes of automatically moving cars or »Go« playing algorithms at world champion level: so-called deep learning processes are almost always behind the success stories. This family of learning methods typically uses overparameterized and usually very large artificial neural networks (DNN) to model the learning problems. The training of such networks requires not only very large amounts of data, but also enormous computing power. Despite the sometimes impressive results achieved with DNNs, they still have some disadvantages that currently often still hinder their broad use in practice: In addition to the typically required very large amounts of data, this is primarily the complex development process.

Design Algorithms Replace Trial and Error

The design of new, problem-specific network topologies is a very time-consuming and computationally intensive process. Until now, the development of new deep learning solutions has been carried out in a purely heuristic and experience-driven "trial-and-error" approach. The aim of the BMBF project "Deep Topology Learning" (DeToL) is to significantly accelerate and simplify this design process for deep learning solutions by means of automatic, data-driven design algorithms.

Next Step: Automation

The field of application of Deep Learning is broad: from machine vision and autonomous driving to speech recognition, music generation or art - Deep Learning is increasingly making a significant contribution to development. At the same time, the number of experts who could design deep neural networks for a specific sub-area is limited.

A logical next step is to involve humans as little as possible in the development of network architectures. Since this automation is very computationally demanding, this is where HPC systems come in. Human interaction is limited to the design of a search space of all possible topologies for a given problem. In the next step, the architecture is then optimized based on a given search strategy.

DeToL's Main Features

  • Automatic design of deep neural networks for given data sets
  • Integration of different search strategies
  • Seamless integration into existing deep learning frameworks
  • Provision of large benchmark data sets from fully trained deep neural networks 
    Hyperparameter optimization and the accurate prediction of performance measures

Project Partners

Five partners, who are international leaders in their respective fields, have joined forces to implement this project: