Seismic Development and processing architecture (SDPA)

Fraunhofer ITWM

The future of programming

The seismic development and processing architecture is the platform to effectively develop and execute seismic applications for todays and tomorrows HPC systems. It is designed for rapid development of fast and scalable applications for large heterogeneous systems.

In the industry, there is not only a constant high demand for improved processing methods but at the same time the need to efficiently use the increasingly larger heterogenous machines of the post-petaflop/s era. We are focusing completely new challenges wrt. scalability, fault tolerance and programmability. Even today there are systems in use with hundred of tousend of cores. Throughput optimization, parallelization and adoption to heterogeneous systems cannot be done by application programmers anylonger. Future systems will even pile on pressure to come up with new solutions.

The goal of the Seismic development and processing architecture is to enable industry users to develop and execute seismic applications within an easy to use graphical environment without knowledge about the concrete underlaying hardware. In the SDPA, we separated coordination from programming. Independently, the HPC experts produces optimized modules while on the coordination layer such modules as well as legacy codes are orchestrated and automatically adopted to and optimized for the concrete hardware.

On the coordination layer, the graphical and hierarchical description of workflows allow to reuse workflow components in many different applications. There are well defined interfaces that enable separate optimization of individual workflow components. Automatic optimizations of workflows, even on-the-fly are supported by an built-in simulator. The simulator collects data like memory consumption or running time. The use of an simulator makes the SDPA independent from the underlying hardware architecture and also ensures the efficient use of future heterogeneous systems with today unknown structure. Also the simulator recognizes certain patterns in the application behaviour and together with the dynamic load balancer the SDPA makes use of it.

The SDPA is built on top of the departements HPC tools and experience. Maximal efficiency in communication and data transfer is achieved by using GPI as central component in the memory virtualization. Individual applications are able to independently access memory from the distributed global memory wich becomes a persistent data storage. Applications use a thin and application independent virtualization layer to access data in the distributed global memory.

Another important aspect is fault tolerance. For todays and even more for tomorrows systems it is neccessary to transparently handle faults. It is not possible to simply restart an application. The SPDA has several mechanisms included like replication of important data or completely independent components. Each component is allowed to fail, nevertheless, the system will finally make progress.

more information