Hello World

The Concept

MPI programs are executed as one or more processes; one process is typically assigned to one physical processor core. All the processes run the exact same program, but by receiving different input they can be made to do different tasks. The most common way to differ the processes is by their rank. Together with the total number of processes, referred to as size, they form the basic method of dividing the tasks between the processes. Getting the rank of a process and the total number of processes is therefore the goal of this example. Furthermore, all MPI related instructions must be issued between MPI_Init() and MPI_Finalize(). Regular C instructions that is to be run locally for each process, e.g. some preprocessing that is equal for all processes, can be run outside the MPI context.

The Code

Below is a simple program that, when executed, will make each process print their name and rank as well as the total number of processes.

$ cd basic_mpi/01_hello_world/
$ cat src/hello.c
/******************************************************************************
*                                                                            *
*  Basic MPI Example - Hello World                                           *
*                                                                            *
*  Find the size of the communicator, your rank within it, and the           *
*  name of the processor you run on.                                         *
*                                                                            *
******************************************************************************
*                                                                            *
*  The original code was written by Gustav at University of Indiana in 2003. *
*                                                                            *
*  The current version has been tested/updated by the HPC department at      *
*  the Norwegian University of Science and Technology in 2011.               *
*                                                                            *
******************************************************************************/
#include   /* printf and BUFSIZ defined there */
#include  /* exit defined there */
#include     /* all MPI-2 functions defined there */
 
int main(argc, argv)
int argc;
char *argv[];
{
int rank, size, length;
char name[BUFSIZ];
 
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Get_processor_name(name, &length);
 
printf("%s: hello world from process %d of %d\n", name, rank, size);
 
MPI_Finalize();
 
exit(0);
}

MPI_Init(); Is responsible for spawning processes and setting up the communication between them. The default communicator (collection of processes) MPI_COMM_WORLD is created.

MPI_Finalize(); End the MPI program.

MPI_Comm_rank( MPI_COMM_WORLD, &rank ); Returns the rank of the process within the communicator. The rank is used to divide tasks among the processes. The process with rank 0 might get some special task, while the rank of each process might correspond to distinct columns in a matrix, effectively partitioning the matrix between the processes.

MPI_Comm_size( MPI_COMM_WORLD, &size ); Returns the total number of processes within the communicator. This can be useful to e.g. know how many columns of a matrix each process will be assigned.

MPI_Get_processor_name( name, &length ); Is more of a curiosity than necessary in most programs; it can assure us that our MPI program is indeed running on more than one computer/node.

Compile & Run

If you have not already done so, obtain all the example code here.

The directory of each example contains two files, job.sh and Makefile in addition to the source folder src which contains the source code.

$ ls
job.sh  Makefile  src

The Makefile is used to describe how the program is to be compiled. The job.sh script is used to describe how the program is to be executed on the system. In addition, the Makefile offers functionality to submit the job to the queue system as well as to clean the directory of temporary files.

Switch to the Intel compiler (optional, only necessary once in each terminal session)

$ module load intel

List other available module

$ module avail

Compile the program using

$ make

Submit the job to the queue

$ make submit

See the queue status

$ qstat

Remove a job from the queue

$ qdel [process_number]

Check the status of a job

$ checkjob [process_number]

The output from the program execution is placed in the output folder

$ cat output/*
compute-0-21.local: hello world from process 2 of 16
compute-0-21.local: hello world from process 0 of 16
compute-0-21.local: hello world from process 1 of 16
compute-0-21.local: hello world from process 3 of 16
compute-0-21.local: hello world from process 4 of 16
compute-0-21.local: hello world from process 5 of 16
compute-0-21.local: hello world from process 6 of 16
compute-0-21.local: hello world from process 7 of 16
compute-0-14.local: hello world from process 9 of 16
compute-0-14.local: hello world from process 8 of 16
compute-0-14.local: hello world from process 10 of 16
compute-0-14.local: hello world from process 11 of 16
compute-0-14.local: hello world from process 12 of 16
compute-0-14.local: hello world from process 15 of 16
compute-0-14.local: hello world from process 13 of 16
compute-0-14.local: hello world from process 14 of 16
Scroll to Top