Research Computing, USF

Linux Tutorial: Chapter 6

6.0 Where to go from here?

Well now that you can do basic commands, move around the system and edit files, you probably want to get started working on the cluster. There are several programs on the cluster to help you get started working. Our software includes gcc, pbs, pgi, and mpich. In this chapter we will introduce how to use these programs.

6.1.0 GCC

6.1.1 What is GCC?

GCC is a shorthand for "GNU Compiler Collection". GCC is a collection of frontends for C, C++, Objective C, Fortran, Java, and Ada. It has most prominately become the standard for compiling C programs under Linux. It is currently controlled by the GCC Steering Committee.

GCC is currently available for use on the cluster, running version 2.91.66. Although you will not usually be compiling your C programs with standard gcc you will need to learn how to program basic C before moving on to MPI programming. GCC provides you with the standard ANSI C libraries needed for you to start.

6.1.2 Compiling programs with `gcc`

To use the C compiler provided by GCC all that is needed is your source code. The GCC compiler translates your source code into object code, readable by the machine, and then links the object code into a binary, executable, file. Lets give you a command-line example:

[bob@mimir c]$ ls -la total 12 drwxrwxr-x 2 bob bob 4096 May 16 16:42 . drwxrwxr-x 3 bob bob 4096 May 16 16:37 .. -rw-rw-r-- 1 bob bob 75 May 16 16:40 hello_world.c [bob@mimir c]$ gcc -o hello_world hello_world.c [bob@mimir c]$ ls -la total 24 drwxrwxr-x 2 bob bob 4096 May 16 16:42 . drwxrwxr-x 3 bob bob 4096 May 16 16:37 .. -rwxrwxr-x 1 bob bob 11757 May 16 16:42 hello_world -rw-rw-r-- 1 bob bob 75 May 16 16:40 hello_world.c [bob@mimir c]$

In this example we have taken the source file hello_world.c and compiled the code into a file that we can run, hello_world. When using the gcc command you will want to specify what the output, executable file, will be called by using the -o argument as we did in the example, the name of the executable file can be anything you want. Then you enter the filename of the source file, which in this example is hello_world.c.

Once the compilation is done we are free to run the program. Let's take a look at the program in action.

[bob@mimir c]$ ./hello_world Hello World! [bob@mimir c]$

As we can see the program runs perfect. You may be wondering why I put a ./ before the program name. This is because the current directory that I am in is not in the $path variable, so I must type the full pathname to run the file, which is /home/bob/programs/c/hello_world. Since the . means the current working directory all I have to do is substitute the . for


/home/bob/programs/c

and then enter the rest of the path, which is simply /hello_world. When combined you get ./hello_world, meaning the same thing as /home/bob/programs/c/hello_world. You have to run all programs that are not in the path by either typing the full path or using this method if you are in the same directory as the executable.

For more information about GCC and using GCC please see the GCC Homepage.

6.2.0 MPICH: A Portable Implementation of MPI

MPICH is a freely available, portable implementation of MPI, the Standard for message-passing libraries. You will be using these libraries to do most of you programming on the cluster. Currently MPICH libraries are for the Fortran 77, Fortran 90, and C/C++ programming languages. MPICH, like GCC will compile a given source file into a executable file, but will compile the program for use with MPI, allowing the program to be run in parallel and take full advantage of the cluster.

6.2.1 Using MPICH

Using MPICH is much like using GCC in the way that the command works. The -o argument is used to specify the name of the output, executable, file. Lets look at an example:

Compiling for C

[bob@mimir mpi]$ mpicc -o greetings greetings.c

Compiling for C++

[bob@mimir mpi]$ mpiCC -o greetings greetings.C or [bob@mimir mpi]$ mpiCC -o greetings greetings.cc

Compiling for Fortran77

[bob@mimir mpi]$ mpif77 -o greetings greetings.f

Compiling for Fortran90

[bob@mimir mpi]$ mpif90 -o greetings greetings.f90

This will create, like GCC, the executable file called greetings. This file will later be scheduled to run with PBS.

For a more extensive look at MPICH's capabilities please visit their homepage.

6.3.0 PGI Compilers

PGI is another set of MPI libraries/compilers that available for use, although we do recommend that you use MPICH. PGI will allow you to compile and link code written in HPF.

6.3.1 Using PGI

Compiling for C

[bob@mimir mpi]$ pgicc -o greetings greetings.c

Compiling for C++

[bob@mimir mpi]$ pgCC -o greetings greetings.C or pgCC -o greetings greetings.cc

Compiling for Fortran77

[bob@mimir mpi]$ pgf77 -o greetings greetings.f

Compiling for Fortran90

[bob@mimir mpi]$ pgf90 -o greetings greetings.f90

Compiling for HPF

[bob@mimir mpi]$ pghpf -o greetings greetings.f90

For more information about PGI please see their online documentation and the official PGI homepage.

6.4.0 PBS Job Scheduler

PBS schedules when programs should be running on the cluster. Some of the programs on the cluster will have a higher priority than others and this is where PBS comes in. PBS will set aside a portion of the cluster for each user to use. The size depends on how many other jobs are currently running and at what priority the other jobs are set at. With PBS we can allow fair distribution of CPU time and memory for all users. To run your MPI enabled program you must submit a job for PBS.

6.4.1 Using PBS

To use PBS we must have an MPI enabled program and a script file to configure the job. Let's get you started by giving you some source code called greetings.c, you can save it to your disk by right-clicking the link and choosing save target as. Now go ahead and compile the program using MPICH.

[bob@mimir mpi]$ mpicc -o greetings greetings.c

Now that we have the executable we need to setup a script for the program to be run with PBS. Right click this link to save the script called greetings.pbs to your disk. If you are doing this on wyrd instead of mimir please use this script. Now make sure that both the greetings and greetings.pbs are in the same directory. We should now be ready to send the job to PBS. Type in the command qsub greetings.pbs you should then see something like this:

[bob@mimir mpi]$ qsub greetings.pbs 1744.mimir.acomp.usf.edu [bob@mimir mpi]$

After some time you should see two files in the directory called greetings.pbs.e1744 and greetings.pbs.o1744. The number at the end of files should match the previous output. The contents of the second file will be your program output, which will be similar to:

Warning: no access to tty (Bad file descriptor). Thus no job control in this shell. Greetings from process 1! Greetings from process 2! Greetings from process 3! Greetings from process 4! ...

You will usually get the first two lines if you are running csh or tcsh, these lines are just warnings and can be ignored. If you got the above lines, congratulations, you just compiled and ran your first MPI program.

Research Computing@USF Blog

Perl-Fu: Automating ILOM SSH Sessions

Ultimate Cluster Monitoring, Part I

Librarian-NG: Solve your unresolved linkage dependancies FAST.

zfs_restore: the cool way to restore from backups

IPMI Monitoring for x4500 & Dell PowerEdge with Nagios

Kickstart and ssh