Compaq Clusters Overview Exercise

  1. Login to the GPS cluster

    Workshops differ in how this is done. The instructor will go over this beforehand.

  2. Copy the example files

    Make sure you are in your home directory, then copy the example codes into a new directory:

    
    cp -r /usr/global/docs/training/blaise/compaq   ./compaq
    
  3. Try the Compaq MPI example:

    1. cd into your ~/compaq/dmpi subdirectory. You should see files for both C and Fortran.
    2. Choose the language you prefer, and then examine the source and the Makefile.
    3. Run make -f [makefile] to compile the example in your preferred language.
    4. Run the executable code using mpirun on 4 processors. For example:
      dmpirun -np 4 mpichk

  4. Try the MPICH example:

    1. cd into your ~/compaq/mpich subdirectory. You should see files for both C and Fortran.
    2. Choose the language you prefer, and then examine the source and the Makefile.
    3. Run make -f [makefile] to compile the example in your preferred language.
    4. Run the executable code using mpirun on 4 processors. For example:
      mpirun -np 4 mpichk

  5. Compare bandwidths:

    1. cd into your ~/compaq/bandwidth subdirectory. You should see files for both C and Fortran.
    2. Depending upon your language preference, compile the source file for MPICH. You can go back to the previous Makefiles in case you aren't sure how.
    3. Run the executable with 2 processes under mpirun. Note the timings.
    4. Now compile the source file for Compaq MPI. Again, you can go back to the previous Makefiles in case you aren't sure how.
    5. Run the executable with 2 processes under dmpirun. Note the timings.
    6. Try running several times - there is a good chance that timings will vary considerably if there are other users on the GPS node you are using (and there probably will be!).
    7. Which performs better in this simple example?

  6. Run a batch job on GPS
    1. cd into your ~/compaq/batch subdirectory.
    2. Review the job command script file called jobscript.gps. Then uncomment one of the compile lines as indicated. ***IMPORTANT*** - if you forget to do this, there will be no executable to run!
    3. Submit the job script to batch: psub jobscript.gps
    4. Check the status of the job until it completes using the pstat -T command.
      NOTE: The time it takes for your job to run is unpredictable and depends upon other users. If you wait for more than a few minutes for your job to complete, move on and come back to this part later.
    5. View the batch output file and compare it to the jobscript. Your output file will be named something like:
      jobscript.gps.oXXXXXX.
      Note: you can ignore the "warning" and "stty" messages at the top of the file.

  7. Login to TC2K

    You can ssh directly to TC2K from your GPS machine.

  8. Check system usage on TC2K
    1. Use the spjstat command and observe the output. Notice the pbatch and pdebug partitions. See the man page if you are curious.
    2. Now try the rinfo command. See the man page for details.

  9. Run interactive on TC2K
    1. cd into your ~/compaq/tc2k subdirectory. You should see files for both C and Fortran.
    2. Choose the language you prefer, and then examine the source and the Makefile.
    3. Run make -f [makefile] to compile the example in your preferred language.
    4. Run the executable code using prun on 4 processors in the pdebug partition. For example:
      prun -n4 -ppdebug mpichk
      Note: Be aware that TC2K is often a heavily used system and that there are only 10 interactive nodes. If your job is rejected, try again when there are available nodes in pdebug (use spjstat to determine this).
    5. Now try the following commands and observe their output to see how -n and -N work together:
      prun -n4 -ppdebug hostname
      prun -n5 -ppdebug hostname
      prun -n4 -N2 -ppdebug hostname
      prun -n5 -N3 -ppdebug hostname

  10. Run a batch job on TC2K
    1. cd into your ~/compaq/batch subdirectory.
    2. Review the job command script file called jobscript.tc2k. Then uncomment one of the compile lines as indicated. ***IMPORTANT*** - if you forget to do this, there will be no executable to run!
    3. Submit the job script to batch: psub jobscript.tc2k
    4. Check the status of the job until it completes using the pstat -T command.
      NOTE: The time it takes for your job to run is unpredictable and depends upon other users. If you wait for more than a few minutes for your job to complete, move on and come back to this part later.
    5. View the batch output file and compare it to the jobscript. Your output file will be named something like:
      jobscript.tc2k.oXXXXXX.
      Note: you can ignore the "warning" and "stty" messages at the top of the file.

  11. Pthreads
    1. Note: apologies to Fortran users - the POSIX threads standard is only defined for the C language.
    2. This can be run on GPS or TC2K - your pick.
    3. cd into your ~/compaq/pthreads subdirectory.
    4. Review the source file - it's simple enough to understand even if you are not familiar with pthreads.
    5. Compile the program: cc -pthread hello.c
    6. Run the executable a.out and observe the output.
    7. Run the executable a few more times and notice the order of the output statements. The output should (although this is not a certainty) differ over several executions - especially on GPS. Why?

  12. OpenMP
    1. This can be run on GPS or TC2K - your pick again.
    2. cd into your ~/compaq/openmp subdirectory. You should see both a C and Fortran version of an omp_hello program.
    3. Review the code - again it should be easy to understand even if you are unfamiliar with OpenMP.
    4. Compile the code using the -omp flag. For example:
      cc -omp omp_hello.c
      f77 -omp omp_hello.f
    5. Run your executable. How many threads were created? By default, OpenMP will create as many threads as there are CPUs on the node.
    6. Now set the OMP_NUM_THREADS environment variable to 6. For example:
      setenv OMP_NUM_THREADS 6
    7. Run the code again. How many threads this time?
    8. As with the pthreads example, run the code several times and notice that the order of output can vary.


This completes the exercise.

Evaluation Form       Please complete the online evaluation form.

Where would you like to go now?