Should this documentation not address an issue with Man2, users should email email@example.com stating that they have a problem on Man2, giving their username on the system (but not their password) and describing the problem.
What is the system most suitable for?
The system is suitable for:
Parallel jobs of moderate size — MPI jobs up to perhaps 16 processes;
- OpenMP jobs up to 16 threads;
- serial jobs, particularly those requiring a lot of RAM (up to 32GB);
- parameter sweeps (using Condor);
- long-running jobs (e.g., up to a week, or longer by arrangement).
What is the system not suitable for?
The system is not suitable for large-scale MPI jobs.
The following software has been installed and tested on Man2:
- Star CD
Intel Fortran 90/95 compiler, ifort
Other software is available under /software but has not been tested.
Man2 has a total of 27 compute nodes:
- 22 with two dual-core Opteron CPUs and 8GB RAM;
- three with two dual-core Opteron CPUs and 16 GB RAM;
- two with eight dual-core Opteron CPUs and 32 GB RAM.
The system has a three TB disk array attached.
The North-West Grid
Man2 is part of the North-West Grid, which consists of HPC clusters from the University of Manchester, the University of Liverpool, Lancaster University and also Daresbury Laboratories.
Getting Access to Man2
University of Manchester academics, postdocs and postgraduates who wish to use Man2 should email firstname.lastname@example.org giving their IT Services username and describing briefly the nature of the nature of their computational work (e.g., serial or parallel, software needed, and, if possible, approximate memory and filesystem capacity requirements).
Those who wish to gain access to all North-West Grid HPC facilities should register via the North-West Grid Web site.
Logging into Man2; File Transfer; X-Windows
There are four things to consider when using Man2:
Making contact with the machine — VPN/Firewall issues.
- Getting an SSH client so that you can login.
- Transfering files to and from the system.
- Enabling the use of GUI-based applications which run on Man2 using X-Windows.
Not all computers can connect to Man2:
- If you have the University VPN enabled on your desktop/laptop you will be able to connect.
- If you are not using the VPN, contact the system administrator to ensure that your computer is given access.
Login Using SSH
Once network issues have been sorted out, as described above, you should be able to login using SSH. First, start the VPN if required.
Linux users will be able to login using OpenSSH, which comes with all popular distros, by typing
ssh -l <username> man2.nw-grid.ac.uk # ...replace <username> with your username, cf. mpciish2...
at the command-line.
MS Windows users should download and install PuTTY, an SSH client.
It is likely that you will wish to upload files to Man2, or download them from Man2 to your desktop/laptop. Linux users can do this by using either scp or sftp, from the OpenSSH utilities suite (which comes will all popular distros), for example:
scp myfile.txt <username>@man2.nw-grid.ac.uk: # ...to copy a file to Man2 --- don't forget the ":" at the end... scp <username>@man2.nw-grid.ac.uk:results.out results.copy # ...copy a file from Man2...
MS Windows users should download and install WinSCP, a GUI-based file-transfer client.
Using X-Windows: GUI-Based Applications
Using SSH on its own will enable you to login to Man2 and use the command-line. If you want to use GUI-based applications, such as gedit, a Notepad-like editor, or you want to use Matlab interactively, then you need to run an X11 server on your local desktop and enable X11-tunnelling in your SSH connection.
All popular Linux distros run an X11-based desktop (GNOME, KDE). The only remaining step is to enable X11-tunnelling when logging in:
ssh -X -l <username> man2.nw-grid.ac.uk # ...that's an UPPERcase X...
MS Windows users will need to download and install an X11 server. The two obvious options are eXceed, for which the University has a site licence, and Xming, which is free to download and install. When connecting:
- Start the VPN, if necessary.
- Start eXceed or Xming, then
start PuTTY, being careful to enable X11 tunnelling — click on "SSH" on the left-hand-side, then ensure the X11 tunnelling box is "checked", before starting a connection to Man2.
Where Everything Is: Filesystems and Backups
The three TB disk array is divided into three parts:
/software, for applications and associated data, which is backed up every day;
/home, for users' home-directories, backed up every day;
/scratch, for additional user storage — this filesystem is not backed up.
Running Jobs on Man2
All computational jobs lasting more than a few minutes must be submitted to one of the two batch systems on Man2: all parallel work (MPI and OpenMP) must be submitted to SGE; serial work can be submitted to either SGE or Condor.
Condor is available as an alternative means of submitting computational jobs to the cluster. It is suitable for serial jobs only; it is well-suited for large numbers of short jobs. Condor is used as backfill on the system; jobs submitted to SGE always have priority.
The following compilers are installed on Man2: gcc, gfortran (GNU's Fortran 90/95 compiler); and Intel's Fortran 90/95 compiler, ifort. Usage details and example qsub scripts for running jobs under SGE are given on the dedicated page.
Open MPI is installed on the system; OpenMPI is an open-source implementation of MPIv2, compiled with GCC (the GNU C compiler) and Ifort (the Intel Fortran compiler). OpenMPI is able to take advantage of multiple physical networks between compute nodes and works well with the batch system, SGE. Information regarding compilation and also submission of jobs to Man2's batch system can be found on the dedicated Man2 OpenMPI page.
N.B. OpenMPI should not be confused with OpenMP which is a shared-memory API, not an implementation of MPI. Libraries:
Now in testing...
Running Stata jobs interactively in the batch system.
R 2.10.1 now available! (2010 March 08.)
Submitting R jobs to the batch system.
Running R interactively, i.e., using the R command line or the GUI.
A recipe for using R for MS Windows users new to Linux and HPC clusters.
Submitting Matlab jobs to the batch system.
Running Matlab jobs interactively.
To use the Matlab2009a via the batch system.
To use the NAG toolbox with Matlab 2009a.
Submitting Castep jobs to the batch system.
Submitting LAMMPS jobs to the batch system.
Submitting Ox jobs to the batch system.
Submitting Gauss jobs to the batch system.
CFD: Fluent, Star-CD and Star CCM+
Submitting Fluent jobs to the batch system.
Running Fluent jobs interactively.
Submitting Star-CD jobs to the batch system.
Submitting Amber v10 jobs to the batch system.
Applications (Not Fully Tested)
export PATH=$PATH:/software/FreeSurfer/bin/ export FREESURFER_HOME=/software/FreeSurfer/ source $FREESURFER_HOME/SetUpFreeSurfer.sh
tkmedit bert orig.mgz
Applications (Not Yet Tested)
The Intel Math Kernel Library is installed under
/opt/intel/mkl/10.0.1.014/lib for version 10 of the Intel compilers
/opt/intel/Compiler/11.0/074/mkl/lib/em64t for version 11 of the Intel compilers
The NAG numerical libraries are installed under /opt/NAG. The current installation includes:
- fll6a22dfl: Fortran 77 Mark 22 library for GNU GFortran v4.3.2 compiler
- fll6i22dcl: Fortran 77 Mark 22 library for Intel v10.1 compiler
- cll6a09dhl: C Mark 9 library for GNU GCC v4.1.2 compiler
- cll6a08dgl: C Mark 8 for GNU GCC and Intel ICC
Usage details and examples can be found on the Man2 NAG Libs wiki page.