HPC Clusters

What Is a High Performance Computing (HPC) Cluster?

A High Performance Computing (HPC) cluster is a collection of multiple servers, known as nodes, which enable the execution of applications. There are usually several various types of nodes in an HPC cluster. HPC software typically runs across hundreds or thousands of nodes to read and write data. Seamless communication is essential to enable communication between the storage system and large numbers of nodes.

HPC is rapidly evolving. We’ll discuss how HPC clusters are changing in response to cloud computing, the growing use of AI applications and models, the availability of powerful data center Graphical Processing Units (GPUs), and the overwhelming adoption of container infrastructure.

This is part of an extensive series of guides about AI Technology.

In this article:

• HPC Cluster Architecture
• HPC in the Cloud
• How HPC Clusters Drive AI Applications
• Best Practices for Modern HPC Workloads
• Design Your Cluster for the Workload
• Containers and HPC
• Leverage GPU Virtualization
• HPC GPU Virtualization with Run:AI

HPC Cluster Architecture

An HPC cluster architecture typically includes the following components:

• Underlying software—an HPC cluster requires an underlying infrastructure to execute applications, as well as software to control this infrastructure. The software used to run an HPC cluster must be able to handle massive volumes of input/output traffic and multiple, simultaneous tasks such as reading and writing data to the storage system from a large collection of CPUs.
• Network switch—an HPC cluster requires high bandwidth and low latency. It is therefore common to use a network switch like Infiniband or a high-performing Ethernet switch.
• Head/login node—this node validates users and allows them to set up software on compute nodes.
• Compute nodes—these nodes perform numerical computations and typically have the highest possible clock rates for the number of cores. These nodes may have minimal persistent storage but high dynamic random-access memory (DRAM).
• Accelerator nodes—these nodes might contain an accelerator (or multiple accelerators), although some applications cannot take advantage of them. In some cases, every node contains an accelerator (for instance, a smaller HPC cluster with a specific purpose in mind).
• Storage system—HPC clusters can use storage nodes or a storage system such as a parallel file system (PFS), which enables simultaneous communication between multiple nodes and storage drives. Powerful storage is important for allowing the compute nodes to operate smoothly and with low latency.

HPC in the Cloud

HPC is usually enabled by a purpose-built cluster for batch processing scientific workloads. You can combine on-premise clusters with cloud computing to handle usage spikes. The cloud is also suited to parallel jobs.

You can implement HPC using a public or private cloud. The major cloud providers allow you to build your own HPC services or leverage bundled services. Certain providers support ready-to-use HPC implementations in a hybrid environment.

Running HPC in the cloud offers the following benefits:

• Distributed workloads—you can use container-based microservices to orchestrate the distribution of workloads more easily in the cloud. Containerization also makes it easier to transfer existing tools and workloads from your on-premise systems.
• Scalable workloads—you can scale resources according to your needs and rapidly increase the processing power, memory, and storage. Scalable workloads reduce bottlenecks and allow you to avoid paying for unutilized resources.
• High availability—the cloud provides high availability, which reduces the risk of interruption to your workloads. High availability leverages multiple data centers, which can enhance data protection.
• Cost savings—leveraging the cloud allows you to avoid the cost of acquiring, storing, and maintaining physical infrastructure. The cloud provider leases the resources you use, minimizing both upfront costs and technical debt. You can save the most for infrequent workloads.

Cloud-based HPC generally requires the following components:

• Instance clusters or virtual machines (VMs)
• Support for bare metal
• Communication with the user space
• Features for batch scheduling
• Fast interconnects
• Storage with low latency and high-throughput

How HPC Clusters Drive AI Applications

Typically, an HPC system contains between 16 and 64 nodes, with at least two CPUs per node. The multiple CPUs ensure increased processing power compared to traditional, single-device systems with only one CPU. The nodes in an HPC system also provide additional storage and memory resources, increasing both speed and storage capacity.

Some HPC systems use graphics processing units (GPUs) to boost processing power even further. You can use GPUs as co-processors, in parallel with CPUs, in a model known as hybrid computing.

HPC can provide the following benefits for developing artificial intelligence (AI) applications:

• Purpose-built processors—GPUs are specialized for efficient AI processing, including algorithms like neural networks.
• Processing speed—co-processing and parallel processing enable fast computation, allowing you to process data and run AI experiments within shorter timeframes.
• Volume of data—the large number of storage and memory resources let you process large volumes of data and run longer analyses, improving the accuracy of your AI models.
• Efficient use of resources—by distributing workloads across the resources available, you can maximize the efficiency of your resource usage.
• Cost savings—an HPC system is a cost-effective way to access supercomputing capabilities. Cloud-based HPC lets you reduce upfront costs with a pay-per-use pricing model for resources.

Best Practices for Modern HPC Workloads

The following best practices can help you make the most of your HPC implementation.

Design Your Cluster for the Workload

Workflow optimization should be a priority when planning your cluster. HPC workloads can differ, so you may want to configure your cluster to optimize the operations of a specific workload.

HPC applications often divide a workload into a set of tasks, which can run in parallel. For some tasks, processing may require more communication between nodes and specialized resources (i.e., hardware/software).

The computational requirements of a workload determine how many compute nodes you need in your cluster, as well as the hardware required for each node and any software you should install. You will also need resources for maintaining security and implementing disaster recovery. To keep the system running, you will need to set up management nodes.

Containers and HPC

HPC workloads are usually monolithic, with HPC applications running large data sets in the data center. Containerized applications offer greater portability, allowing you to package HPC applications and run them across multiple clusters to handle large data sets.

Containerization enables a microservices architecture that can benefit application services. Building applications in a microservices environment typically involves packaging small services into containers. You can manage the lifecycle of each service independently, based on the specific granular scaling, patching, and development requirements of the service.

HPC application workloads can also benefit from the isolated management of development and scaling. The scalability of containers enables HPC workloads to handle spikes in data processing requirements without downtime. Containerized HPC applications can scale easily to accommodate these spikes.

Additional advantages of containers used in a microservices architecture include modularity and speed. You can use containers with multiple software components, dependencies, and libraries to run complex HPC applications. Containers help break down the complexity of HPC workloads and facilitate deployment.

Leverage GPU Virtualization

Your GPU implementation has a significant impact on your HPC workloads. GPU providers typically offer virtualization software that can GPUs available to multiple computing resources.

HPC workloads typically require a single compute instance mapping to one or more GPUs. These configurations depend on technologies that allow VMs or bare metal machines to communicate directly with physical GPUs running on other machines.

HPC GPU Virtualization with Run:ai

Run:ai automates resource management and orchestration for HPC clusters utilizing GPU hardware. With Run:ai, you can automatically run as many compute intensive workloads as needed.

Here are some of the capabilities you gain when using Run:ai:

• Advanced visibility—create an efficient pipeline of resource sharing by pooling GPU compute resources.
• No more bottlenecks—you can set up guaranteed quotas of GPU resources, to avoid bottlenecks and optimize billing.
• A higher level of control—Run:ai enables you to dynamically change resource allocation, ensuring each job gets the resources it needs at any given time.

Run:ai simplifies HPC infrastructure, helping teams accelerate their productivity and conserve costs by running more jobs on fewer resources.

Learn more about the Run:ai GPU virtualization platform.

See Our Additional Guides on Key AI Technology Topics

Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of AI Technology.

Deep Learning for Computer Vision

• TensorFlow CNN: Building Your First CNN with Tensorflow
• PyTorch ResNet: The Basics and a Quick Tutorial
• Understanding Deep Convolutional Neural Networks

Deep Learning GPU

• Best GPU for Deep Learning: Critical Considerations for Large-Scale AI
• PyTorch GPU: Working with CUDA in PyTorch
• Top 8 Deep Learning Workstations: On-Premises and in the Cloud

Machine Learning Engineer

• Machine Learning Infrastructure: Components of Effective Pipelines
• Machine Learning Automation: Speeding Up the Data Science Pipeline
• Machine Learning Workflow: Streamlining Your ML Pipeline

MLOps

• Apache Airflow: Use Cases, Architecture, and Best Practices
• Edge AI: Benefits, Use Cases & Deployment Models
• JupyterHub: A Practical Guide

Multi GPU

• Keras Multi GPU: A Practical Guide
• PyTorch Multi GPU: Four Techniques Explained
• Tensorflow with Multiple GPUs: 5

NVIDA A100

• NVIDIA Deep Learning GPU: AI & Machine Learning Guide
• NVIDIA DGX: Under the Hood of DGX-1, DGX-2 and A100
• NVIDIA NGC: Features, Popular Containers & Quick Tutorial

CUDA NVIDIA

• CUDA Programming: An In-Depth Look
• CUDA vs OpenCL
• NVIDIA cuDNN: Fine-Tuning GPU Performance for Neural Networks