Linux Cluster General Usage

Logging in to the cluster

Login via ssh from any of our systems to apollo.sns.ias.edu (the master, or head node).

Once you are on apollo, the master node, compile and debug your code, (or perform data analsys).

Once you have executable code to run, submit a job to the cluster using PBS as documented here.

If you need to, you can connect to the other nodes, node2,node3,node4 ..... node33, with rsh e.g. rsh node14. All nodes can see your cluster home directory /home/username.

Scratch Space

50GB of local scratch space is available under /scratch on each node. 400GB of global scratch space is available under /nettmp.

Create a directory e.g. /scratch/username and use this to store temporary files/data in a more organised way.

Because of the file permissions on /scratch and /nettmp, users can't delete each others files. The /scratch and /nettmp directories are cleaned nightly to purge files older than 10 days.

Input/Output Issues

The best way to make data portable is to use a self-describing data format such as HDF (Hierarchical data format). This avoids both classes of possible I/O problems described below.

Endianness

Endianness concerns the size and order of bytes in unformatted data. Our desktop Linux workstations and the cluster share the same little-endian byte order so endianness will only be an issue if you would like to make use of unformatted data from another architecture such as Sparc (which is big-endian).

For code written in Fortran, the PGI compilers on the cluster accept the -byteswapio option when compiling in order to accept big-endian data. If you have code written in c, or are using the gcc compiler, you'll need to byteswap the data as described here.

64 vs 32 bit

The cluster is 64 or 32 bit capable vs 32 bit on the deskstops. As such if you want to input unformatted data from our desktops you need to do it with 32 bit code, which doesn't use the cluster to it's greatest potential.

Compile 32 bit code with gcc as follows:

gcc -m32 file.c
Compile 32 bit code with PGI as follows: 
pgcc -tp px file.c

If you output unformatted data on the cluster resulting from a 64 bit process you won't be able to read it on on our 32 bit desktop systems. Save formatted output to use the data elsewhere, or output a structured data format such as HDF instead.