Automatically Differentiating MPI-1: The Complete Story
by
Alan Carle
Rice University, Department of Computational and Applied Mathematics
Coauthors: Mike Fagan (Rice University)

A popular way to add parallelism to a sequential programming language is to use a library that defines routines for communication of data via ``message-passing.'' The MPI-1 (Message Passing Interface Version 1.0 and 1.1) library provides a standardized message-passing system for writing portable message-passing programs. MPI-1 provides a wide-range of facilities including point-to-point communication, collective communication, reduction operations, and process synchronization for programs written in Fortran 77 or C. MPI provides primitive datatypes representing floating point and integer data, and a collection of constructor operations for hierarchically creating derived datatypes.

There is a clear need to provide automatic differentiation for programs written using the MPI message-passing library. Odyssée  and Adifor 3.0 both provide minimal support for MPI. Odyssée provides forward and reverse mode support for synchronous SEND, RECV, and REDUCE operations for primitive MPI datatypes. Adifor 3.0 additionally supports the asynchronous versions of SEND and RECV, and a few other operations, also limited to the primitive datatypes. We discuss the steps we have taken to extend Adifor 3.0 to support the complete MPI-1 standard, including the MPI derived datatype constructors that introduce a notion of pointer-based structured data into Fortran. Specifically, we describe the implementation of a portable interface to the MPI-1 library and changes to Adifor 3.0 to support differentiation of MPI programs. Future extensions to support MPI-2 will be suggested.

Automatically Differentiating MPI-1: The Complete Story

Date received: December 30, 1999