Nag.com

Chapter Z01
Library Utilities
Contents
Scope of the Chapter
Background to the Problems
Managing a Logical Processor Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PVM-based Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MPI-based Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Processor Grid Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PVM-based Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MPI-based Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Workspace Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Relationship to PVM and the BLACS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Relationship to MPI and the BLACS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recommendations on Choice and Use of Available Routines
Selection of Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Programming Advice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction – Z01
Scope of the Chapter
This chapter provides utilities that simplify the task of safely creating and terminating parallelapplications using the NAG Parallel Library. The Library assumes the Single Program Multiple Data(SPMD) model of parallelism (see [4]) and uses a logical grid of processors defined by the Basic LinearAlgebra Communications Subprograms (BLACS, see [1]). This Chapter also provides routines that returninformation appropriate to the message-passing system being used such as the MPI rank (see [3]) or thePVM task identifiers (see [2]) of the logical processors. Routines in this chapter also provide a convenientway to determine the workspace requirements for some of the numerical routines included in the Library.
No examples of the use of these routines are given in this chapter because their use is illustrated in theexample programs for other routines in the NAG Parallel Library.
Background to the Problems
The aim of Chapter Z01 is to provide a mechanism for allowing novice users to quickly develop parallelprograms while allowing experienced users the flexibility to develop more sophisticated programs. Sincemanaging the processor grid (Library Grid) and obtaining processor grid information differ in the PVM-based version and the MPI-based version of the NAG Parallel Library, this chapter provides separatesections that describe the ways in which a Library Grid is set up and operated upon in a PVM or an MPIenvironment. Hence, users should read the sections appropriate the version of the Library being used.
Managing a Logical Processor Grid
Exception to a few routines (as described in the relevant routine documents), users should set up a‘Library Grid’ prior to a call to the NAG Parallel Library routines. The underlying mechanism in whichthis Library Grid is created is different in the MPI-based and PVM-based versions of the Library.
PVM-based Version
The most important use of the routines in this chapter is to set up a rectangular grid of logical processorsexecuting identical programs (according to the SPMD model). A simplified mechanism allows users toexecute a single program and a sufficient number of identical copies of this program are automaticallyexecuted (spawned) to form the requested processor grid. However, users also have the option of creatingprocesses themselves and simply using Chapter Z01 utilities to declare a logical processor grid for useby NAG Parallel Library routines, although this option should not be used unless the user is familiarwith PVM and the BLACS. This allows the user to multigrid (an example is given in the NAG ParallelLibrary Tutorial, PVM-based Version). Note that in the current release of the NAG Parallel Library,only one Library Grid is permitted. When the logical processor grid has been defined, NAG ParallelLibrary routines may be called.
The BLACS assume a static physical machine; i.e., the number of processors never varies. In keeping withthis assumption, the first call to the Z01 spawning utility, Z01AAFP, creates enough logical processors toform the requested grid, and this number never changes throughout the execution of the program. Theuser is allowed to reshape the grid (i.e., vary the dimensions of the grid) subject to the condition thatthe total number of processors never exceeds the number initially created.
A processor grid is uniquely identified by a ‘context’ for the purpose of communications and globaloperations within the grid. A context should be thought of as a pointer to a processor grid; its value neednever be examined and should never be modified by the user. A processor which belongs to a particularlogical grid is said to be ‘in context’ for that grid. Processors which are not in context continue to the nextsynchronisation point in the program (a point in the code that all processors must reach together beforecontinuing execution). This synchronisation point will usually be where a processor grid is terminated orreshaped.
When the user has completed calls to NAG Parallel Library routines or wants to reshape the processorgrid, the existing grid should be ‘undefined’ using the Z01 utility routine Z01ABFP. The user mustindicate through an argument to Z01ABFP whether further Library Grids will be declared or if therewill be no more calls to library routines.
PVM offers the user some alternative options in the way that tasks are spawned. The simplified spawningmechanism provided in the NAG Parallel Library only duplicates one of these alternative PVM options, Introduction – Z01
for use in debugging parallel programs. The other spawning options that PVM provides can be utilised ifthe users choose to spawn tasks themselves and simply use the NAG Parallel Library utilities for declaringthe logical processor grid.
MPI-based Version
The most important use of the routines in this chapter is to set up a rectangular grid of logical processorsexecuting identical programs (according to the SPMD model). A simplified mechanism allows users toexecute a single program concurrently on a processor grid. However, users also have the option of creatingprocesses themselves and simply using Chapter Z01 utilities to declare a logical processor grid for useby NAG Parallel Library routines, although this option should not be used unless the user is familiarwith MPI and the BLACS. This allows the user to multigrid (an example is given in the NAG ParallelLibrary Tutorial, MPI-based Version). Note that in the current release of the NAG Parallel Library, onlyone Library Grid is permitted. When the logical processor grid has been defined, NAG Parallel Libraryroutines may be called.
The BLACS (and also MPI) assume a static physical machine; i.e., the number of processors nevervaries. In keeping with this assumption, the first call to the Z01 utility, Z01AAFP, claims enough logicalprocessors to form the requested grid. However, the user is allowed to reshape the grid (i.e., vary thedimensions of the grid) subject to the condition that the total number of processors never exceeds thenumber initially created by MPI.
A processor grid is uniquely identified by a ‘context’ for the purpose of communications and globaloperations within the grid. A context should be thought of as a pointer to a processor grid; its value neednever be examined and should never be modified by the user. A processor which belongs to a particularlogical grid is said to be ‘in context’ for that grid. Processors which are not in context continue to the nextsynchronisation point in the program (a point in the code that all processors must reach together beforecontinuing execution). This synchronisation point will usually be where a processor grid is terminatedor reshaped. A call to Z01AAFP also returns a context for the Library Grid and all Library routines usethis context for safe communication. However, facilities are provided which allow users to use a contextcreated independently in the Library routines (an example is given in the NAG Parallel Library Tutorial,MPI-based Version).
When the user has completed calls to NAG Parallel Library routines or wants to reshape the processorgrid, the existing grid should be ‘undefined’ using the Z01 utility routine Z01ABFP. The user mustindicate through an argument to Z01ABFP whether further Library Grids will be declared or if therewill be no more calls to library routines.
Processor Grid Information
PVM-based Version
Each processor in the logical processor grid is identified by the coordinates of its position in the processorgrid. The size of the processor grid and the position of each processor can be determined by the BLACSroutine BLACS GRIDINFO, but there is additional information that may be relevant for using theNAG Parallel Library. In particular, it is often necessary to identify the ‘root’ processor since this isthe processor that is used to access standard input and standard output. If the NAG Parallel Librarymechanism for spawning the processor grid is used, then the ‘root’ processor is always labelled {0,0} inthe grid.
It is also useful to know which processors are ‘in context’ for the library routines and which are not, sinceit is possible (but not recommended for novice users) to give the out-of-context processors other usefulwork (but not to call additional NAG Parallel Library routines).
Additional information from the underlying PVM system can be obtained, in particular the task identifiersof each logical processor in the grid. This allows communication between logical processors using directcalls to PVM routines, although the use of the BLACS for inter-processor communication is stronglyrecommended.
The most efficient logical topololgies for broadcasts and global operations using the BLACS are alsoreturned by Z01 routines.
Introduction – Z01
MPI-based Version
Each processor in the logical processor grid is identified by the coordinates of its position in the processorgrid. The size of the processor grid and the position of each processor can be determined by Z01BGFP.
In particular, it is often necessary to identify the ‘root’ processor since this is the processor that is usedto access standard input and standard output. If the NAG Parallel Library mechanism for creating theprocessor grid is used, then the ‘root’ processor is always identified by the process with rank 0 (this isusually labelled as {0,0} in the grid). The NAG Parallel Library provides a routine which returns thecoordinates of the root processor in the logical processor grid.
It is also useful to know which processors are ‘in context’ for the library routines and which are not, sinceit is possible (but not recommended for novice users) to give the out-of-context processors other usefulwork (but not to call additional NAG Parallel Library routines).
The most efficient logical topololgies for broadcasts and global operations using the BLACS are alsoreturned by Z01 routines.
Workspace Functions
Some of the routines in the NAG Parallel Library require a workspace array which must be suitablydimensioned. Typically the minimum size of the array depends on the size of the problem, the size of thelogical processor grid and occasionally, the position of a logical processor within the grid. The size of theworkspace is usually defined in terms of functions which are implemented in Chapter Z01.
Relationship to PVM and the BLACS
It is possible to make calls to PVM and BLACS routines from a program which uses the PVM-basedversion of the NAG Parallel Library. The PVM task identifiers returned by Z01BDFP and the BLACScontext returned by Z01AAFP may be utilised to make direct calls to PVM and BLACS routines. Seethe NAG Parallel Library Tutorial (PVM-based version) for an example.
Relationship to MPI and the BLACS
It is possible to make calls to MPI and BLACS routines from a program which uses the MPI-basedversion of the NAG Parallel Library. The MPI rank returned by Z01BGFP and the BLACS contextreturned by Z01AAFP may be utilised to make direct calls to MPI and BLACS routines. However, ifMPI routines are being called, users should first create an MPI communicator from the BLACS contextfor safe communication in their Library Grid. This can easily be achieved by a call to the BLACS routineBLACS GET. See the NAG Parallel Library Tutorial (MPI-based version) for an example.
References
[1] Dongarra J J and Whaley R C (1995) A users’ guide to the BLACS v1.0. LAPACK Working Note
94 (Technical Report CS-95-281) Department of Computer Science, University of Tennessee, 107Ayres Hall, Knoxville, TN 37996-1301, USA.
URL: http://www.netlib.org/lapack/lawns/lawn94.ps [2] Geist A, Beguelin A, Dongarra J J, Jiang W, Manchek R and Sunderam V (1994) PVM: Parallel
Virtual Machine. A Users’ Guide and Tutorial for Networked Parallel Computing The MIT Press,Cambridge, MA, USA [3] Gropp W, Lusk E and Skjellum A (1994) Using MPI: Portable Parallel Programming with the
Message-Passing Interface MIT Press, Cambridge, MA [4] McBryan O A (1994) An overview of message passing environments Parallel Comput. 20 417–444
Recommendations on Choice and Use of Available Routines
Note: Refer to the Users’ Note for your implementation to check that a routine is available.
Introduction – Z01
Selection of Routine
Grid Management
Purpose
’Undefine’ a processor grid or terminate library usage Inform library mechanism of a user-spawned grid in PVM environment (only available in the PVM-based version of the Library)Inform library mechanism of a user-created grid in MPI environment (only available in the MPI-based version of the Library)Spawn debug sessions in PVM environment (only available in the PVM-based version Grid Information
Identify ‘root’ processor
Return PVM information (only available in the PVM-based version of the Library) Return MPI information (only available in the MPI-based version of the Library) Workspace Functions
Number of rows/columns of a cyclic 2-d block distribution held locally
Workspace size for F08AEFP (PDGEQRF), F08AFFP (PDORGQR), F08ASFP (PZGEQRF) and F08ATFP (PZUNGQR)Workspace size for F08AGFP (PDORMQR) and F08AUFP (PZUNMQR) Identify the processor row/column that holds a matrix row/column in the cyclic 2-d Programming Advice
The user must be careful to program according to the Single Program Multiple Data model of parallelism.
It is possible to mimic other models of parallelism from within the SPMD model but this should be donewith caution.
Examples of the use of the routines in this chapter are given for each numerical routine document andalso in both versions of the NAG Parallel Library Tutorials. It is strongly recommended that you readthe Tutorial before attempting to use the Library.

Source: http://www.nag.com/numeric/FD/manual/pdf/Z01/z01_intro_fd02.pdf