BRAID

Next-generation high-performance language interoperability

Date: 2013-02-01
Version: 0.2.6
Manual section:1
Manual group:Compilers and Programming Languages

SYNOPSIS

braid [-h] [--gen-sexp] [--gen-sidl] [-c <language>] [-s <language>]
[--makefile] [-i] [--suppress-contracts] [-m <prefix>] [--debug] [--profile] [--version] [--license] [-v] [<file.sidl> [<file.sidl> ...]]

DESCRIPTION

Braid 0.2.6 - Do magically wonderful things with SIDL (scientific interface definition language) files. BRAID is a high-performance language interoperability tool that generates Babel-compatible bindings for the Chapel programming language. For details on using the command-line tool, please consult the BRAID man page and the Babel user's guide.

About BRAID and its relation to Babel

logos/braid-logo-small.png

BRAID, the Braid system for Rewriting Abstract Descriptions, is a compiler-like tool to generate the glue code necessary for language interoperability. Developed by the Components project [1] at Lawrence Livermore National Laboratory, BRAID supports the Scientific Interface Definition Language (SIDL) for the language-independent declaration of interfaces associated with scientific software packages.

The language bindings generated by BRAID are binary-compatible to those generated by the related Babel [2] tool. While Babel handles traditional programming languages (such as C, C++, Fortran (77-2008), Python, and Java), BRAID adds support for modern PGAS [3] languages.

In practice, this means that if you want to, for example, let a Chapel program use a component implemented in Fortran 95, you would invoke Babel for the Fortran code and Braid for Chapel code:

braid --makefile --client=Chapel interface.sidl
babel --makefile --server=F95    interface.sidl
[1]https://computation.llnl.gov/casc/components/
[2]https://computation.llnl.gov/casc/components/
[3]http://en.wikipedia.org/wiki/Partitioned_global_address_space

Supported Languages

At this moment Chapel client code is well-tested and considered stable. Chapel server works also fairly well, however, the Chapel compiler imposes the limitation that there can only be one copy of the Chapel runtime system per executable (e.g., 1 Chapel library or one Chapel main() function).

Language Client Server
Chapel stable stable
UPC testing planned
X10 planned planned

In addition to all regular Babel/SIDL features, the Chapel backend also extends the Chapel runtime to support borrowed arrays; i.e., Arrays that are initialized by an external function and can be passed to Chapel code without copying. It also provides an interface for distributed arrays which allow non-Chapel code to access non-local data residing in the PGAS-space.

The following features are not yet implemented:

Generic arrays, arrays of objects, structs, contracts, hooks and RMI.

Installation

BRAID uses an autotools-based build system, so the regular:

./configure && make install

will work just fine. Below is the step-by-step break-down of the the

installation process:

tar xvf braid-0.2.3.tar.bz2 && cd braid-0.2.3
mkdir build && cd build
../configure --prefix=<INSTALL_PATH>
make [-j<N>] && make install

# other useful commands
../configure --help
make check

Compiling Chapel to run on SLURM-managed Clusters with Infiniband

We found the following setup to be working with MPI (to spawn processes) and direct Infiniband (for the actual communication).

It seems that Chapel (~1.5) cannot be compiled with a GCC version newer than 4.6.X, because of issues with GASnet and the Chapel code base. A known good version of GCC for compiling Chapel is 4.3.6. To compile Chapel in CHPL_HOME use the following commands:

export CHPL_COMM=gasnet
export CHPL_COMM_SUBSTRATE=ibv
make -C ${CHPL_HOME}

To initialize the environment (using bash) run:

export CHPL_COMM=gasnet
export CHPL_COMM_SUBSTRATE=ibv
cd ${CHPL_HOME}
source util/setchplenv.bash
cd -

To run a Chapel program using Infiniband communication and MPI to spawn new processes via SLURM:

export MPIRUN_CMD="srun -p<partition> -N %N %P %A" # you can customize this
./program

Those are substitutions made by Chapel (or GASnet), here is a list taken from one of the Chapel-internal Makefiles:

# The following substitutions are performed:
#   %P = program executable name
#   %N = requested node count
#   %A = program arguments
#   %Q = program arguments w/ an extra level of quotes
#   %D = the current working directory
#   %H = hostfile (if any)
CONDUIT_RUNCMD = mpirun -np %N %P %A

The way I understand it SLURM provides the mpirun launcher on the machines it manages. Chapel/GASnet uses this mechanism to trick SLURM into launching processes on multiple nodes. If you compiled Chapel with the 'ibv' conduit, however, no actual communication will be done over MPI, instead GASnet will directly use the Infiniband network to exchange data between nodes.

Using Infiniband directly is supposed to perform better because MPI apparently does not do one-sided communication well.

Patching the Chapel compiler

You will need to patch your Chapel 1.4.0 compiler using the following command _after_ running ./configure:

bash <builddir>/patch-chapel-compiler.sh

The script will patch and recompile the compiler automatically. You can undo this operation by running:

bash <builddir>/patch-chapel-compiler.sh --undo

this will reverse the effects of the patch.

User-visible dependencies

If you just want to compile and install BRAID, you will need: - Python: Version 2.6 or higher - gcc, ld, Perl, AWK, sed

If you want to run the regression tests, you will also need: - GNU make Version 3.74 or higher - Babel: Version 2.0 or higher - Chapel: Version 1.4.0 - Java: JVM 1.6 or higher - NumPy: Version 1.0.4 or higher

Developer-only dependencies

  • Make: GNU make version 3.74 or higher
  • Autotools: Version 2.65 or later
  • SWI-Prolog: Version 5.10.4 or higher (only needed if you intend to modify [ir,sidl].def)
  • Doxygen: Version 1.6 or higher (disable with ./configure --disable-documentation)
  • graphviz: (for Doxygen)
  • GNU flex
  • rst2man: Contained in python-docutils (Debian, Ubuntu, openSUSE and Fedora)

User's Guide

Anatomy of a Babel call

A call from one language into another starts with the client (the caller) invoking a stub, which accepts all the arguments of the method in the native data format (such as value representation or memory layout) of the client language. The stub is automatically generated by Braid/Babel and is usually output in the client's language, or in C, if the native language is not expressive enough to perform the argument conversion, which often involves byte-level memory manipulation. Stubs are very small and the compiler can sometimes inline them. The stub converts the arguments into the intermediate object representation (IOR) which is Babel's native data format. It then calls the server implementation, which also has to accept the IOR. On the server side resides a skeleton, which does the reverse operation of converting the IOR into the native representation of the server (callee). The skeleton invokes the implementation which is the actual method to be called. Upon return, it converts all outgoing arguments and the return value into the IOR and returns to the Stub, which performs the translation back to the client's data format.

Scientific Interface Definition Language (SIDL)

Braid cannot (yet) parse source code. In order to generate bindings for a component the user needs to specify the interface that the component provides in the SIDL format. A lengthy description of all SIDL features can be found in the .. _Babel Manual: https://computation.llnl.gov/casc/components/docs/users_guide/ , Chapter 6.

Chapel Language Binding

This section describes how the various features of SIDL are mapped onto Chapel constructs.

Introduction

Chapel is a modern high-level parallel programming language originally developed by Cray Inc. as part of the DARPA HPCS program. In contrast to traditional programming languages such as C++, the runtime system of the language takes care of executing the code in parallel. The language still offers the user fine-grained control over things such as the data layout and the allocation of distributed data, but it renders the tedious explicit encoding of communication through, e.g., a message-passing interface, obsolete.

Basics

To see examples of how to use the Chapel language with Braid, please refer to the many examples located in the braid/regression/interop directory. The corresponding SIDL files can be found in the Babel sources the regression subdirectory.

Static functions

Chapel classes cannot have static methods. BRAID will generate a <classname_>_static module that contains all static methods defined by the SIDL class classname.

Object lifecycle

Objects are created using the <classname_>_static.create(out exception) function. This indirection is necessary because constructors may throw a SIDL exception. The <classname_>_static.wrap_<classname>(in obj, out exception) function can be used to create a Chapel version of an existing externally created SIDL object.

Objects can be up-casted by using the provided as_<baseclass>() methods. Down-casting can be down by using the generated <baseclass>_cast_<target>() methods.

Scalar datatypes

The following table lists the scalar types supported by SIDL and the corresponding Chapel types used by the skeleton or stub while converting Chapel code from or into the IOR. The SIDL scalar types are (with the exception of strings) of fixed length and were easy to support especially since Chapel has parametric support for the number of bits in the integral and floating point types which map to the same representation as used by the IOR. It also has native types for both single-precision and double-precision complex numbers and supports opaque types that allow data to be passed around through Babel/BRAID back into the original address space. Chapel also supports enumerated type to defines a set of named constants. On the other hand, the Babel IOR and the Chapel compiler use different representations for complex numbers, hence BRAID generates glue code to pass around copies. Since Chapel does not have a char type, BRAID generates code to convert Chapel unit-length strings into chars using a statically allocated lookup table.

SIDL type Size (in bits) Corresponding Chapel type
bool 1 bool
char 8 string (length=1)
int 32 int(32)
long 64 int(64)
float 32 real(32)
double 64 real(64)
fcomplex 64 complex(64)
dcomplex 128 complex(128)
opaque 64 int(64)
string varies string
enum 32 enum

Scalar Data Types in SIDL and their Chapel equivalents on a 64-bit machine.

Array data types

SIDL arrays can be generated by using the sidl.<type>_array.create() family of functions:

var max_size = 1024;
var arr = sidl.double_array.create2dCol(max_size, max_size);
var a_meta = arr(1)
var a_data = arr(2)
for i in 0..#max_size do {
  for j in 0..#max_size do {
    a_data[i:int(32),j:int(32)] = 3.14;
  }

The create functions return a tuple with SIDL-related metadata and a Chapel borrowed array. Borrowed arrays are an extension to Chapel; they inherit all the syntactic sugar support Chapel provides for natively defined arrays. Hence there is no change in the Chapel code while using these arrays except during array creation. We require a small extension to the Chapel compiler to support borrowed arrays. Borrowed arrays have data blocks allocated external to the Chapel runtime unlike traditional Chapel arrays where each array uses a data block managed by the Chapel runtime. This avoids superfluous allocation and deallocation of array data blocks while passing the array reference between Babel/BRAID calls. It becomes the user's responsibility to manage the memory while using borrowed arrays.

In the above example, a generic SIDL array<> version of the created SIDL array could be accessed using the a_meta.generic data member. Generic arrays are passed as Chapel opaque values. To cast them to a proper sidl.Array, use the following code:

proc generic_array_input(in ga: opaque /* array< > */) {
  var ga_meta = sidl.double_array.cast(ga);
  if ga_meta != nil then {
    var ga_data = createBorrowedArray1d(ga_meta);
    ga_data[0] = 42;
  }
}

From the user's perspective there is no difference between Chapel representation of SIDL arrays and R-arrays. It should be noted that non column-major arrays will be automatically copied and converted in the Chapel stub generated by BRAID.

Exceptions

Chapel does not (yet) support exception handling. To throw an exception, a Chapel method should create a new exception object and assign it to the _ex argument before returning.

Client-side

Implementation-side

UPC Language Binding

This section describes how the various features of SIDL are mapped onto Unified Parallel C constructs.

Braid so far has been tested with Berkeley UPC v. 2.14.2.

TODO.

Development status

BRAID is written in 98% Python; the SIDL scanner is implemented in flex (C). Some of the Python sources are automatically generated from a high-level specification (sidl.def, ir.def) by a Prolog script. The implementation language choice is motivated by Python being the highest-level language that we can assume to be pre-installed on all our target systems. So far we have three components:

  • A complete parser for SIDL which generates an object-oriented intermediate representation (IR)

  • A converter to an extensible s-expression-based language independent IR

  • Code generators that convert this IR into Chapel and C code. Other languages supported by Babel will follow.

    To facilitate the writing of these code generators we put some effort into extending Python with a pattern-matching mechanism for arbitrarily complex tuples. (And the s-expressions from the IR are internally represented as Python tuples.)

This diagram shows the work-flow implemented in BRAID:

             Parser               Conversion
+-------------+  +------------------+  +---------------------+
| SIDL        |--| SIDL-based       |--| Language indep. IR  |
|             |  | declarative IR   |  | (s-expressions)     |
+-------------+  +------------------+  +---------------------+
                                         |              |   |
                                         |   Code       |   |
                                         |   Generators |   |
                                         |              |   |
                                       +-----------+ +----------+
                                       | Chapel    | | C, ...   |
                                       |           | |          |
                                       +-----------+ +----------+

The idea to use the s-expression-based IR to interface with ROTE at some later point. Other components (e.g. PAUL) can also generate this IR to get access to the code generators.

Further Information

The following files are available at the top of the release directory structure provide additional information on the Babel release:

Authors

Active Members

Adrian Prantl: Architect, main author adrian@llnl.gov, adrian@complang.tuwien.ac.at

Summer Interns

Shams Imam, Rice University

Please report bugs to <components@llnl.gov>. consult the BRAID man page and the Babel user's guide.

OPTIONS

positional arguments

<file.sidl> SIDL files to use as input

optional arguments

-h, --help show this help message and exit
--gen-sexp generate an s-expression
--gen-sidl generate SIDL output again
-c <language>, --client <language>
 generate client code in the specified language (Chapel, UPC, or any language supported through Babel)
-s <language>, --server <language>
 generate server code in the specified language (Chapel, UPC, or any language supported through Babel)
--makefile generate a default GNUmakefile
-i, --generate-hooks
 generate pre-/post-method hooks
--suppress-contracts
 refrain from generating contract enforcement from SIDL specs
-m <prefix>, --make-prefix <prefix>
 <prefix> is prepended to the name of babel.make and the symbols defined internally to allow Braid to be run multiple times in a single directory.
--debug enable debugging features
--profile enable profiling
--version print version and exit
--license print licensing details
-v, --verbose print more debug info

Please report bugs to <components@llnl.gov>.