For most of the platforms, there are two
available: 32-bit and 64-bit.
This table specifies valid TARGET names for various supported
|Sun ultra||SOLARIS||SOLARIS64||64-bit version added in GA 3.1|
|IBM BlueGene/P||BGP||supported in GA 4.1 and later (Contact your BlueGene sys admin for GA instalation). More info in support page...|
||added in GA 4.0.2 (Contact your
BlueGene sys admin for GA instalation). More info in support page...
|TARGET based on the OS in compute Nodes (Catamount/Linux). More info and sample settings in support page...|
|IBM RS/6000||IBM||IBM64||64-bit version added in GA 3.1|
|IBM SP||LAPI|| LAPI64
||no support yet for user-space communication in the 64-bit mode by IBM|
|Compaq/DEC alpha||not available||DECOSF|
|HP pa-risc||HPUX||HPUX64||64-bit version added in GA 3.1|
x86, ultra, powerpc
x86_64 (Opteron), ppc64, etc
|Linux alpha||not available||LINUX64||64-bit version added in GA 3.1;
Compaq compilers rather than GNU required
|Cray T3E||not available||CRAY-T3E|
|Cray J90||not available||CRAY-YMP|
|Cray SV1||not available||CRAY-SV1|
|| not available
||In X1, by default, TARGET
is defined by the operating system as cray-sv2
|SGI IRIX mips||SGI_N32, SGI||SGITFP|
|Hitachi SR8000||HITACHI||not available|
|Fujitsu VPP systems||FUJITSU-VPP||FUJITSU-VPP64||64-bit version added in GA 3.|
|NEC SX series
|| Running MAC X or higher
To aid development of fully portable applications, in 64-bit mode Fortran integer datatype is 64-bits. It is motivated by 1) the need of applications to use very large data structures and 2) Fortran INTEGER*8 not being fully portable. The 64-bit representation of integer datatype is accomplished by using appropriate Fortran compiler flag.
Because of limited interest in heterogenous computing among known us
GA users, the Global Array library still does not support
platforms. This capability can be added if required by new
|Network||Protocol name||ARMCI_NETWORK setting||Supported platforms|
|Ethernet||TCP/IP||SOCKETS (optional/default)||workstation clusters (32 and 64-bit)|
|Quadrics/QsNet||Elan3/Shmem|| QUADRICS or ELAN3
||Linux (alpha,x86,IA64,..), Compaq|
| Quadrics/QsNet II
||Linux (32 and 64-bit)|
||OPENIB|| Linux (32 and 64-bit).
NOTE: This network is supported in GA versions>=4.1. For more info see support page...
||Linux (32 and 64-bit)|
|Giganet cLAN||VIA||VIA|| Linux (32 and 64-bit)
|MPI||MPI-SPAWN||Supported in GA 4.1 or higher. This network setting can be used on any platform that has MPI-2 dynamic process management support. Using this setting is recommended only if your network is not listed above.|
The recent versions of MPICH (an MPI implementation from ANL/Mississippi State) keep the MPI header files in more than one directory and provide compiler wrappers that implicitly point to the appropriate header files. One can :
On Windows NT, the current version of GA was tested with WMPI, an NT implementation derived from MPICH in Portugal.
GA may also depend on other software depending on the functions being used.
The GA program should look like:
.... do work .... do work
tidy up global arrays
call mpi_finalize() MPI_Finalize() ! tidy up MPI
stop ! exit program
.... do work .... do work
! tidy up global arrays
call pend() PEND_() ! tidy up tcgmsg
stop ! exit program
The ma_init call looks like :
status = ma_init(type, stack_size, heap_size)
and it basically just goes to the OS and gets stack_size+heap_size elements of size type. The amount of memory MA allocates need to be sufficient for storing global arrays on some platforms. Please refer to section 3.3.1 for the details and information on more advanced usage of MA in GA programs.
Compiler Settings (optional): For various supported platforms, the default compilers and compiler options are specified in config/makefile.h. One could change the predefined default compilers and compiler flags in GA package either by specifying them on the command line or in the file config/makefile.h. Note: editing config/makefile.h for any platform requires extra care and intended for intermediate/advanced users.
gmake FC=f90 CC=cc FOPT=-O4 COPT=-gNote that GA provides only Fortran-77 interfaces. To use and compile with a Fortran 90 compiler, it has to support a subset of Fortran-77.
Example: using csh/tcsh (assume using MPICH installed in /usr/local on IBM workstation)
setenv USE_MPI y
setenv MPI_LOC /usr/local/mpich
setenv MPI_LIB $MPI_LOC/lib/rs6000/ch_shmem
setenv MPI_INCLUDE $MPI_LOC/include
with TCGMSG: To build GA directly with TCGMSG, the user must
define the environmental variable
Note: When MSG_COMMS=TCGMSG, make sure to unset the environment variable USE_MPI (e.g. unsetenv USE_MPI).
After chosing the configuration, to build the GA library, type
make or gmake
If the build is successfull, a test program test.x will be created in global/testing directory. Refer Section "Running GA programs" on how to run this test.
To build an application based on GA located in
for example, the application's name is app.c
(or app.F, app.f), type
make app.x or gmake app.x
Please refer to compiler flags in file g/config/makefile.h
to make sure that Fortran and C compiler flags are consistent with
use to compile your application. This may be critical when Fortran
flags are used to change the default length of the integer datatype.
Interface to ScaLAPACK: GA interface routines to ScaLAPACK are only available, when GA is build with MPI and ScaLAPACK. Before building GA, the user is required to define the environment variables USE_SCALAPACK or USE_SCALAPACK_I8 (for scalapack libraries compiled with 8-byte integers), and the location of ScaLAPACK & Co. libraries in the env variable SCALAPACK.
Example: using csh/tcsh
setenv USE_SCALAPACK y
setenv SCALAPACK '-L/msrc/proj/scalapack/LIB/rs6000
-lscalapack -lpblas -ltools -lblacsF77cinit -lblacs'
setenv USE_MPI y
Since there are certain interdependencies
blacs and blacsF77cinit, some system might require specification of
twice to fix the unresolved external symbols from these libs.
GA C++ Bindings: By default, GA C++ bindings are not built. GA++
is built only if GA_C_CORE is defined as follows:
setenv GA_C_CORE y
make clean; make (This will build GA with C core and C++ bindling).
GA_C_CORE: GA's internal core is implemented using Fortran and
C. When GA_C_CORE is set, core Fortran
functionalites are replaced by their C counterparts to eliminate the
hassle involved in mixing Fortran and C
with C++ bindings on certain platforms or for some compilers (like, missing Fortran symols/libraries during the
linking phase). NOTE: C and C++ compilers should be
from the same family. GA_C_CORE doesnot support mixing C and C++
compilers (e.g.using Intel compiler for C and GNU compiler for C++).
make FC=ifort CC=icc CXX=g++ (not supported if GA_C_CORE is set)
make FC=ifort CC=icc CXX=icpc (Intel compiler family - supported)
First of all, one needs to set environment variables (same as in Unix enviroment). GA needs to know where find the MPI include files and libraries. To do this, select the Environment tab under the Control Panel, then set the variables to point to the location of MPI, for example for WMPI on disk D:
set MPI_INCLUDE as d:\Wmpi\IncludeMake sure that the dynamic link libraries required by the particular implementation of MPI are copied to the appropriate location for the system DLLs. For WMPI, copy VWMPI.dll to \winnt.
set MPI_LIB as d:\Wmpi\Console
In the top directory do,
nmakeThe GA test.exe program can be built in the g\global\testing directory:
nmake test.exeIn addition, the HPVM package from UCSD offers the GA interface in the NT/Myrinet cluster environment.
GA could be built on Windows 95/98. However, due to the DOS shell limitations, the top level NTmakefile will not work. Therefore, each library has to be made separately in its own directory. The environment variables referring to MPI can be hardcoded in the NT makefiles.
2.3.3 Writing and building new GA programs
For small programs contained in a single file, the most convenient approach is to put your program file into the g/global/testing directory. The existing GNU make suffix rules would build an executable with the ".x" suffix from any C or Fortran source file. You do not have to modify makefiles in g/global/testing at all. For example, if your program is contained in myfile.c or myfile.F and you place it in that directory, all you need to do to create an executable called myfile.x is to type: make myfile.x .
Windows nmake is not as powerful as GNU make - you would need to modify the NT makefile.
This approach obviously is not feasible for large packages that contain multiple source files and directories. In that case you need to provide apropriate definitions in your makefile:
Example: If you are using
MPICH (or MPICH-like Implementations), and mpirun requires a machinefile or hostfile, then
run the GA program same as any other MPI programs. The only
change required is to make sure the hostnames are specified in a
consecutive manner in the machinefile. Not doing this, will
prevent SMP optimizations and would lead to poor resource utilization.
mpirun -np 4 -machinefile machines.txt app.x
Contents of machines.txt: (Let us say we have two 2-way SMP nodes (host1 and host2, and correct formats for a 4-processor machinefile is shown in the below table)
host1 (This is wrong. Same hosts should be specified together
If app.x is built based on TCGMSG (not including, Fujitsu, Cray J90, and Windows, because there are no native ports of TCGMSG), to execute the program on Unix workstations/servers, one should use the 'parallel' program (built in tcgmsg/ipcv4.0). After building the application, a file called 'app.x.p' would also be generated (If there is not such a file, make it: make app.x.p). This file can be edited to specify how many processors and tasks to use, and how to load the executables. Make sure that 'parallel' is accessible (you might copy it into your 'bin' directory). To execute, type: