Workflows for Distributed Multidiscipline Optimization 

Workflows systems are the focus of many e-Science applications. Among the most popular systems are Taverna, Kepler, Pegasus, Bonita and many others. They complement scientific software environments like Dakota, Scilab and Matlab in their ability to provide complex application factories that can be shared, reused and evolved. Further, they support the incremental composition of hierarchic composite applications. Providing a control flow approach, they also complement the usual dataflow approach used in programming toolboxes. Another bonus is that they provide seamless user interfaces, masking technicalities of distributed, programming and administrative layers, thus allowing the users and experts to concentrate on their areas of interest.


Optimization testcases using YAWL, Matlab, Python, Scilab, OpenFOAM and Paraview (OMD2 project).

The OPALE project is investigating the use of the YAWL workflow management system for distributed multidiscipline optimization. The goal is to develop a resilient workflow system for large-scale optimization applications. It is based on extensions to the YAWL system to add resilience and remote computing features on high-performance distributed infrastructures. This includes large-PC clusters connected to broadband networks. It also includes interfaces with the Scilab scientific computing toolbox and the ProActive middleware. A prototype implementation is underway for the OMD2 project ("Optimisation Mutlidiscipline Distribuée") supported by the French Agence Nationale de la Recherche (ANR).


YAWL interfaces.

Provided as an open-source software, YAWL is written in Java. It is based on an Apache server using Tomcat and Apache's Derby relational database system. YAWL is developed by the University of Eindhoven (NL) and the University of Brisbane (Australia). It runs on Linux, Windows and MasOS platforms. It allows complex workflows to be defined and supports high-level constructs (e.g., XOR- and OR-splits and joins, loops, conditional control flow based on application variables values, composite tasks, parallel execution of multiple instances of tasks, etc) through high-level user interfaces. Formally, it is based on a sound and proved operational semantics calling on the "workflow patterns" of the Workflow Management Coalition and implemented by colored Petri nets.


YAWL architecture.

Designed as a open platform, YAWL supports interactions with external and existing software and application codes written in any programming languages, as well as distributed computing.
It includes a native Web Services interface, custom services invocations through "codelets", as well as rules, powerful exception handling facilities, and monitoring of workflow executions.
Further, it supports dynamic evolution of the applications by extensions to the existing workflows through "worklets", i.e., on-line inclusion of new workflow components during execution.
It supports automatic and step-by-step execution of the workflows, as well as persistence of (possibly partial) executions of the workflows for later resuming, using its internal database system.
It also features extensive event logging for later analysis, simulation, configuration and tuning of the application workflows.
Additionnally, YAWL support extensive organizations modeling tools allowing complex projects and users to be defined with adequate access rights and granting capabilities to the various workflows and processing tools.

screen12.png screen10.png
Optimization (2D testcase) using YAWL, Matlab and OpenFOAM (OMD2 project).
Testcase courtesy Renault, ECP, CD-Adapco and SIREHNA (March 2010).

Current experiments (March 2010) include industrial testcases, involving the connection of the Matlab, Scilab, Python, ParaView and OpenFOAM software to the YAWL platform. The YAWL workflow system is used to define the otpimization processes, include the testcases and control their execution: this includes reading the input data (StarCCM+ files), the automatic invocation of the external software and automatic control passing between the various application components (e.g., Matlab scripts, OpenFOAM, ParaView, ...).

Optimization (3D testcase) using YAWL, Python and OpenFOAM (OMD2 project).
Testcase courtesy Renault, ECP, CD-Adapco and SIREHNA (March 2010).

Further developments will include the interface with the ProActive middleware (2010), deployment on a distributed architecture including Grid5000 (2011), and support for resilience, i.e., application-level fault-tolerance (2011).
We wish to thank MM. Ludovic Bellier, Jean-Marc Joseph and Guillaume Lelaurain, engineers for the Service des Moyens Informatiques (Computer and Software Support Group) from the INRIA Grenoble Rhone-Alpes Research Center for their expertise and advice that helped us to deploy the testcases and the running environment.

Optimization (3D testcase) using YAWL, ParaView, Python, OpenFOAM and data converted from Star-CCM+ (OMD2 project, March 2010).

Cas28.jpg Cas29.jpg
Testcase courtesy Renault, ECP, CD-Adapco and SIREHNA (March 2010).
Parameter sweeping in YAWL (OPALE project, April 2010).

Distributed Optimization Platform.

Wing profile optimization workflow (OPALE project, 2009).

  Virtual computing environments 

The advantages demonstrated by grid-computing infrastructures and complex problem solving environments are fundamental in terms of raw computing power and massive storage media. Indeed, they allow parallel and distributed computing of demanding applications that can now be deployed on thousands of connected processors and they can connect petabytes of data through tens gigabits/sec networks at affordable cost.

So far, however, there seems to be some reluctance from the industry to use these environments because they require expertise in the computer science field which is currently not yet widely available on the job market. Users have become in the last decade experts in the manipulation of Web browsers to access inter-continental mass of data and execute remotely located pieces of code transparently. Unfortunately, grid computing environments are still far from providing these seamless and flexible interfaces. New concepts and interfaces are therefore required to alleviate these shortcomings.

Supersonic wing profile optimisation by project OPALE.

One approach that is currently being explored by the OPALE project is the definition of virtual computing infrastructures by which users will be able to define their specific computing environment and use it with their own ad-hoc procedures. This requires the design and implementation of powerful middleware services implemented on top of existing grid environments. The goal is to provide standardised services and the corresponding procedures to help the users specify the resources and computing environment they need to run the complex (and soon, multidisciplinary) applications they currently execute. This implies the design of generic graphic problem solving interfaces, of the implementation of enabling middleware and of ad-hoc interfaces on top of existing grid environments. An international consortium has been set up on this subject, including twelve corporate and academic research institutions, as well as industry partners from Italy, Great Britain, Greece, Cyprus and France. The Opale project is one of the founding members of this consortium.

The CAST platform has also been ported on the Sophia Antipolis cluster and several other Linux workstations in Grenoble with the purpose of developing a grid computing environment for biomedical applications with A. Habbal in Sophia Antipolis. An anti-angiogenesis simulation has thus been parallelised by Jean Arnaud and is being ported on this grid computing environment.

Aerodynamics wing optimization tesbed using genetic algorithms.

  CAST software integration platform

The main contributions concerning the software platform CAST are twofold :

The first aspect implied the parallelisation of the original code using MPI and the use of the PC-clusters and other various Linux workstations at INRIA Sophia Antipolis and Grenoble. This work was implemented by Jean Arnaud. It also resulted in the deployment of the CAST distributed software platform on this environment.

The second aspect concerning Virtual Computing Environments resulted in the setup of an industrial collaboration with CNES for the design of seamless application design interfaces on the Grid.

Further, strong implication in European Networks, e.g., Promuval, led us to specify with European aircraft manufacturers and research centers the characteristics of a software integration platform for multidisciplinary code validation and verification in aeronautics. A demonstator of this platform is currently being designed in cooperation with Alenia Aeronautica (Italy) and Cenaero (Belgium).

Web site for CAST and  CAST user's manual.

  CAST user interface: the aerodynamics wing optimization workflow on the VTHD grid.