Computational Science Community Wiki

Star-CD on Mace01

New service

Overview

Prerequisites

Star-CD uses SSH to start MPI-related processes. Therefore you will need to ensure you have promptless, passwordless SSH access across the Mace01 cluster to run parallel Star-CD jobs. This is done using an SSH key and an appropriate known_hosts file.

New users will have the required SSH configuration set up for them automatically. Established users may not have the required configuration. If in doubt, do not hesitate to ask the system-administrator for help.

MPICH vs LAM

Star-CD comes with a choice of MPI implementations to use. On Mace01, problems have been experienced with MPICH, which was initially chosen, when running jobs with more than two processes (i.e., using more than one compute node). Initial tests indicate that using LAM may lead to fewer problems — hopefully none.

Submitting a Parallel Star-CD Job to the Batch System, SGE

Star-CD jobs should be submitted to either the parallel-R2.q queue or the parallel-R5.q queue. (The remaining parallel queue, parallel-R4.q, is reserved for software which can take advantage of the dedicated MPI network on Rack 4.)

This is an experimental qsub script which depends on an experimental SGE parallel environment. It has been tested by only the system-administrator only (2009/Feb/20).

  #!/bin/bash

  #$ -S /bin/bash
  #$ -cwd

  #$ -q parallel-R2.q
      # ...or parallel-R5.q...

  #$ -pe starcd.pe 4
      # ...this is a custom-built SGE parallel environment...

  export LM_LICENSE_FILE=1999@130.88.124.202
  export STARINI=Default

  MACHINEFILE="machinefile.$JOB_ID"

  # ...choose EITHER...
  #
  # . /software/starcd_402_001/etc/setstar
  #
  # ...OR...
  #
  . /software/starcd_402_001_lam/etc/setstar

  #
  # ...use EITHER starcd.pe-generated machinefile (from SGE's PE_HOSTFILE) :
  #
  star -dp -nodefile=$MACHINEFILE
  #
  # ...OR...use Star-CD auto-detection of SGE env :
  #
  #star -dp $PNP_JOBNODES

  exit_on_error $?