2.5 BUILDING INTEL TRACE ANALYZER
(VAMPIR) INSTRUMENTED GLOBAL ARRAYS
Contents
- New functions needed for the instrumentation
- Build instructions
- Further information
- Known problems
New functions needed for the
instrumentation
- To instrument the GA three C-functions are defined (see
g/ga_vt.c):
- vampir_symdef is defined to associate integer identifiers with
user defined states and activities. It handles any errors that might
occur.
- vampir_begin is defined to register entering a user defined
state. It uses a global counter called <vampirtrace_level> to
avoid tracing the use of libraries within libraries.
- vampir_end is defined to register leaving a user defined state.
The interfaces of these functions are defined below.
void vampir_symdef
(int id, char *state, char *activity, char *file, int line);
void vampir_begin
(int id, char *file, int line);
void vampir_end
(int id, char *file, int line);
In addition to these functions two functions are defined to initialise
and finalise MPI when needed. The use of MPI is required because
Vampirtrace uses it internally. The functions are
void vampir_init
(int argc, char **argv, char *file, int line);
void vampir_finalize
(char *file, int line);
If the cpp flag -DMPI is provided then these two functions will turn
into null functions. In that case the use of MPI within the GAs
will ensure that Vampirtrace will be initialised properly.
The values for <file> and <line> are substitute with
__FILE__ and __LINE__ macros. On compilation the C-preprocessor
replaces these macro with the actual file name and line number. These
values are used to generate
error messages if needed. These functions are defined in the file
g/ga_vt.h.
For each of the instrumented libraries an initialisation routine must
be defined that sets the state and activity tables up.
- tcgmsg: tcgmsg_vampir_init in
g/tcgmsg/tcgmsg_vampir.c. This routine
is called from
within PBEGINF.
- tcgmsg-mpi:
tcgmsg_vampir_init in g/tcgmsg-mpi/tcgmsg_vampir.c called
from ALT_PBEGIN_
in misc.c.
- armci:
armci_vampir_init in g/armci/src/armci_vampir.c called from
ARMCI_Init in
armci.c.
- global: ga_vampir_init in
g/global/src/ga_vampir.c called from
ga_initialize_
and ga_initialize_ltd_ in global.armci.c
Build instructions
To build GA with Vampir (now called, Intel Trace Analyzer) set the
environment variable GA_USE_VAMPIR.
e.g. setenv GA_USE_VAMPIR y
to compile the GAs including all the Vampirtrace instrumentation.
Further environment variables that are required are
LIBVT : The name of the library,
normally -lVT which is the default.
VT_LIB : The path to the library,
-L<library-path> e.g. setenv VT_LIB /usr/local/vampir/lib
VT_INCLUDE: The path to the include file VT.h,
-I<include-path>. e.g. setenv VT_INCLUDE /usr/local/vampir/include
On some platforms it may be necessary to set LIBMPI to -lpmpi to load
the MPI profile interfaces that vampirtrace needs.
There are no defaults for VT_PATH and VT_INCLUDE. Beyond this point
simply follow the GA make instructions.
Note that libVT.a should be loaded before mpi or pmpi otherwise the
vampirtracing will be ignored.
Further information
More information on using Intel Trace Analyzer can be found on the
Intel website at
http://www.intel.com/software/products/cluster/tanalyzer/
From this location Vampir and Vampirtrace can be downloaded for various
platforms including validation licenses if needed.
Known problems
- Vampirtrace and LAM-MPI clash
In an attempt to produce traces while running with LAM-MPI the program
would always abort in MPI_Init due to a segmentation violation. The
Pallas
website does not mention LAM-MPI at all, but does explicitly state that
Vampirtrace does work with MPICH. Indeed the latter has been confirmed
in tests.
Therefore it is not recommended to use the Vampirtrace instrumentation
with LAM-MPI.