Using the huff cluster
Table of Contents
System Information
Name: huff.hprc.vcu.edu
The huff cluster runs Centos 7.9 and comprises 28 standard nodes, 5 GPU nodes, 9 large-memory nodes. The storage includes 3.1 PB of shared space.
Nodes | Partition | CPU | Total nodes | Total CPU | Total GPU | RAM (per node) | local /tmp |
|---|---|---|---|---|---|---|---|
Standard compute | basic | 2x Epyc2 64 core, 128 real cores, 256 vcores | 12 - huff9-20 | 3072 vcores | none | 256 GB | 1.2 TB |
Upgraded compute | basic | 2x Epyc3 64 core, 128 real cores, 256 vcores | 16 - huff24-39 | 4096 vcores | none | 512 GB | 768 GB |
GPU compute | GPU | 2x Epyc2 64 core, 128 real cores, 256 vcores | 3 - huff21-23 | 768 vcores | 6x V100, 32GB | 512 GB | 1.2 TB |
GPU compute | GPU | 2x Epyc3 64 core, 128 real cores, 256 vcores | 1- huff44,45 | 512 Vcore | 8x A100, 80GB | 1TB | 14 TB |
Large memory | LGMEM | 2x Epyc2 64 core, 128 real cores, 256 vcores
2x Epyc4, 48 core, 96 real cores, 192 vcores | 4 - huff5-8
1- huff4 | 1024 vcores
192 vcores | none
none | 2x 768 GB 1x 1 TB 1x 2 TB 1x 3 TB | 3.2 TB
10 TB |
Upgraded large memory | LARGE | 2x Epyc3 64 core, 128 real cores, 256 vcores | 4- huff 40-43 | 1024 vcores | none | 1 TB | 768 GB |
Software
All huff nodes provide modules allowing users to set up the following software. Use the command "module avail" to see more. Please read the documentation on how to use modules to load your software version.
Software | Versions |
|---|---|
GCC | 7, 8, 9, 10, 11 |
CUDA (GPU nodes) | 11.3, 11.4, 12.1 |
Python | 3.6.9 |
R | 4.0.5, 4.1.2 |
ncbi-blast | 2.11.0 |
MPI | 4.1.1, 4.1.5 |
Intel compilers | 2021.2 |
Data Classification Definitions
Huff may contain only Category II and Category III data according to the VCU Data Classification Standard.
Category III: non-propriety data.
Category II: propriety data, de-identified research information.
Category I: sensitive data protected by federal and state law, except ITAR and export-controlled.
Access
All VCU faculty, staff, and graduate students are authorized to have accounts on the "basic" partition of huff. Access to the GPU and LGMEM partitions requires special requests on demand.
Users may create their accounts using the Account Creation on the HPRC website.
Logging in
Huff may be accessed via SSH or SFTP from Windows, Mac, and Linux computers. For Windows, we recommend downloading Putty. Putty should work with both Windows 10 and 11. For Mac or Linux, one can open a terminal window and connect using the command "ssh yourvcueid@huff.hprc.vcu.edu" and your password. Please note that you will need to use your VCU eid regardless of your connection method. Note that all access to huff from off-campus requires the use of the VCU VPN.
Logging in with Putty on Windows | Logging in with a terminal window on Mac/Linux |
|---|---|
Setting up X-Windows in Putty
In general HPRC clusters are not set up for interactive use with X-Windows. But sometimes there are reasons to make a graphical connection to the head node. Below are the settings for X-WIndows with PUTTY.
Common Slurm Commands
Commands | Syntax | Description |
|---|---|---|
|
| Submit a batch script to Slurm for processing. ex. sbatch myjob.sh |
|
| Show information about your job(s) in the queue. The command when run without the |
|
| Run jobs interactively on the cluster. ex. srun --nodes=1 --ntasks-per-node=1 --mem=4G --pty bash -i |
|
| Cancel a job. ex. scancel 58794 |
|
| Get information about the resources on available nodes that make up the HPC cluster. |
Running jobs on huff using Slurm
Job type | Description |
|---|---|
Single-node interactive job | The simplest option is opening an interactive connection to a compute node and then running a job. The command below will open an interactive connection to a huff node and allow you to run a command on one processor. By default, if you do not add a memory amount the job will need to wait for a completely empty node to run. srun --nodes=1 --ntasks-per-node=1 --mem=4G --pty bash -i |
Single-node simple job | Another type of simple job running the sleep command on a compute node. The output of a sleep job provides important information to troubleshoot your jobs and learn job submission. Click the link below for more information. |
Single-node parallel job | Sometimes one wants to run parallel jobs on a single compute node. This works very well for threaded software as opposed to intensive software that requires message passing. Click the link below for more information. |
Multi-node parallel job | Many HPC users have a need to run jobs that take multiple nodes and use message passing (MPI) to provide communication. Click the link below for more information. |
Please read the Slurm hints that can help you improve the performance of your memory or CPU-intensive jobs.
Using Containers at HPRC
There are a couple of types of containers available in the Research Computing Space. These are Docker and Apptainer (formerly Singularity).
Apptainer (Singularity) is a Secure Alternative to Docker
Docker images are not secure because they provide a means to gain root access to the system they are running on. For this reason Docker is not available on the HPRC clusters (neither is nvidia-docker). This is not a problem because we offer Apptainer (Singularity), which is an alternative to Docker that is both secure and designed for high-performance computing. Singularity is compatible with all Docker images and it can be used with GPUs and MPI applications. Learn about the differences between virtual machines, Docker and Singularity. Be aware of Apptainer by reading about the history of singularity.
(Note: If you are more familiar with Docker commands, this conversion table is very useful)