Computational Science Community Wiki

RedQueen2

Redqueen2 (RQ2),

  rq2.rcs.manchester.ac.uk

is a HPC cluster administered by RCS and built on a shared facility model: research groups buy compute nodes (and other hardware) which are added to the cluster; all members of any such group may use the system.

In July 2010, some of the older compute nodes within the cluster were replaced with newer, more computationally powerful nodes via the Revolving Green Fund. So while the system is primarily for those who have invested in compute nodes, limited access is now available for all computational researchers on campus.

RedQueen2 runs Scientific Linux v5.x (a build from source, with modifications, of RedHat Enterprise Linux, cf. CentOS).

The primary role of this system is to run batch (i.e., non-interactive) jobs in the traditional way. A limited number of jobs may also be run interactively (i.e., by using an application's GUI).

Further Help

Should this documentation not address an issue with RedQueen2, users should email its-research@manchester.ac.uk stating that they have a problem on RedQueen2: please give your username on the system (but not your password) and describe the problem.

Help for Users New to Linux-Based HPC Clusters

This documentation assumes a basic knowledge of Linux-based HPC clusters, for example, how to access the system 'via' SSH and X-Windows (aka X11), and the concept of a batch system and queues. If you are new to these research computing and are not familiar with these things, please read this introductory material.

System Hardware

Currently (2010 October) the system comprises:

There is no fast interconnect.

What is the system most suitable for?

The system can be used for any research-related computation including:

As there is no fast interconnect (e.g., no Myrinet or Infiniband), the system is not suitable for large-scale parallel work.

Available Software

Applications

Are to be found under /software.

Compilers

  /opt/intel/Compiler/11.0/074/bin/intel64/icc
  /opt/intel/Compiler/11.0/083/bin/intel64/ifort

and

  /opt/intel/Compiler/11.1/064/bin/intel64/ifort

Maths Libraries

MPI Implementations

    openmpi-1.4.1-gcc44-gfortran44 
    openmpi-1.4.1-intel-11.0 

Getting Access to RedQueen2

To get an account on RedQueen2, email its-research@manchester.ac.uk .

Logging into RedQueen2

Log in using SSH, after starting the VPN if required.

Linux users will be able to login using OpenSSH, which comes with all popular distros, by typing

    ssh -l <username> rq2.rcs.manchester.ac.uk
        # ...replace <username> with your username, cf. mpciish2...

at the command-line.

MS Windows users should download and install PuTTY, an SSH client.

File Transfer

It is likely that you will wish to upload files to RQ2, or download them from RQ2 to your desktop/laptop. Linux users can do this by using either scp or sftp, from the OpenSSH utilities suite (which comes will all popular distros), for example:

    scp myfile.txt <username>@rq2.rcs.manchester.ac.uk:
        # ...to copy a file to RQ2 --- don't forget the ":" at the end...

    scp <username>@rq2.rcs.manchester.ac.uk:results.out results.copy
        # ...copy a file from RQ2...

MS Windows users should download and install WinSCP, a GUI-based file-transfer client.

Using X-Windows: GUI-Based Applications

Using SSH on its own will enable you to login to RQ2 and use the command-line. If you want to use GUI-based applications, such as gedit, a Notepad-like editor, or you want to use Matlab interactively, then you need to run an X11 server on your local desktop and enable X11-tunnelling in your SSH connection.

All popular Linux distros run an X11-based desktop (GNOME, KDE). The only remaining step is to enable X11-tunnelling when logging in:

    ssh -X -l <username> rq2.rcs.manchester.ac.uk
        # ...that's an UPPERcase X...

MS Windows users will need to download and install an X11 server. The two obvious options are eXceed, for which the University has a site licence, and Xming, which is free to download and install. When connecting:

  1. Start the VPN, if necessary.
  2. Start eXceed or Xming, then
  3. start PuTTY, being careful to enable X11 tunnelling — click on "SSH" on the left-hand-side, then ensure the X11 tunnelling box is "checked", before starting a connection.

Running Jobs on Redqueen2: SGE

All computational jobs lasting more than a few minutes must be submitted to the batch system, SGE. Both interactive (e.g., GUI-based) and non-interactive (i.e., batch) jobs can be run, though we 'strongly' discourage the use of interactive jobs as such jobs use more system resources than the equivalent batch jobs.

The documentation linked for each application listed below contains examples illustrating how to submit batch jobs to SGE.

SGE Configuration: Available Queues

Since this is a relatively small system, there is only one available queue

    mixed.q

which can be used for serial, parallel and interactive work.

SGE Configuration: Available Parallel Environments

Three parallel environments are available: for Fluent jobs, please use fluent.pe; for OpenMPI-based jobs please use either orte.pe or orte-12.pe. The former allows any size of parallel job; with the latter, the size of a parallel jobs must be a multiple of 12 — this ensure only whole compute nodes are used.

User Filesystems and Backups

Backups

Home directories are backed up to the IT Services central system.

Applications, Compilers and Libraries

Home Directories

Users' home-directories are in /home-fs1-b1, /home-fs1-g1, /home-fs2-b1 and /home-fs2-g1.

Scratch Space

Scratch space for users is available in /scratch-fs1-f1, /scratch-fs1-h1, /scratch-fs2-f1 and /scratch-fs2-h1. Scratch space is NOT backed up in any way.

Using the Installed Software with the Batch System (SGE): Details and Examples

Matlab

Both serial and parallel Matlab jobs may be submitted to the batch system; Fluent may also be run interactively — though the latter is strongly discouraged, except for short, test jobs:

Star CD

Star CCM

Fluent

Both serial and parallel Fluent jobs may be submitted to the batch system; Fluent may also be run interactively — though the latter is strongly discouraged:

Code Aster

Documented on the School of MACE CFD Group's wiki pages.

Code Saturne

Documented on the School of MACE CFD Group's wiki pages.

Gaussian