NVIDIA DGX Station

NVIDIA DGX Station 100: Technical Specifications

The NVIDIA DGX Station A100 has the following technical specifications:

• Implementation: Available as 160 GB or 320 GB
• GPU: 4x NVIDIA A100 Tensor Core GPUs (40 or 80 GB depending on the implementation)
• CPU: Single AMD 7742 with 64 cores, between 2.25 GHz and 3.4 GHz
• Performance: 2.5 petaFLOPS AI and 5 petaOPS INT8
• RAM: Up to 512 GB of DDR4 memory
• Storage: 4x NVMe drives for high-speed local storage, with optional storage expansion, including a 1.92 TB NVMe drive for operating system storage, 7.68 TB for internal, and a U.2 NVMe drive

• Networking: Single-port 1Gbase-T Ethernet, dual-port 10Gbase-T Ethernet LAN, BMC management port
• Power consumption: 1500W at 100-120 VAC
• Dimensions: 25.1 x 10.1 x 20.4 inches (63.9 x 25.6 x 51.8 cm)
• Weight: ~90 pounds (32 kg)
• Operating temperature range: 41–95 ºF (5–35 ºC).

5 Use Cases of NVIDIA DGX Station

The NVIDIA DGX Station A100 is a powerful computing system that is well-suited for a wide range of AI and ML workflows. Some common use cases for the DGX Station A100 include:

1. Natural language processing (NLP): The DGX Station A100 can be used to train and deploy models for NLP tasks such as language translation, sentiment analysis, and text generation.
2. Computer vision: The DGX Station A100 is well-suited for training and deploying models for computer vision tasks, including object recognition, image classification, and video analysis.
3. Speech recognition: The DGX Station A100 can be used to train and deploy models for speech recognition tasks, such as transcribing audio and converting speech to text.
4. Predictive modeling: The DGX Station A100 can be used to train and deploy predictive models for a variety of applications, including forecasting, risk assessment, and recommendation systems.
5. HPC: The DGX Station A100 can also be used for high-performance computing (HPC) applications, such as simulations and data analytics.
   

In addition to the use cases listed above, the DGX Station A100 can be used for a wide range of other AI and ML workflows, including machine translation, robotics, and bioinformatics.

NVIDIA DGX Station A100 System Architecture

NVIDIA A100 GPU

The NVIDIA A100 Tensor Core GPU is the central component of the NVIDIA DGX Station A100 system architecture. The A100 GPU is a powerful accelerator that is designed specifically for AI, ML, and HPC applications. It is based on the NVIDIA Ampere architecture and includes a range of features that make it well-suited for these types of workloads.  For example, it includes the third-generation NVLink, which is a high-speed interconnect that allows the GPU to communicate with other components in the system, such as the CPU and other GPUs. This allows the DGX Station A100 to scale to multiple GPUs, providing even more performance for AI and ML workloads.

In the NVIDIA DGX Station A100 system architecture, the A100 GPU is responsible for accelerating AI, ML, and HPC workloads. It works in conjunction with the CPU and other components in the system to provide fast and efficient performance for these types of workloads.

Third-Generation Tensor Cores

Tensor Cores are specialized hardware units that are designed to accelerate AI and ML workloads. They are a key feature of the NVIDIA A100 Tensor Core GPU, which is the central component of the NVIDIA DGX Station A100 system architecture.

Third-generation Tensor Cores are the latest generation of Tensor Cores, and they are included in the A100 GPU. They are designed to be even more efficient and powerful than previous generations of Tensor Cores, and they offer several key benefits:

• High-speed matrix multiplication and convolution: Third-generation Tensor Cores can perform matrix multiplication and convolution operations at high speeds, which are critical for training and running deep learning models. This allows the A100 GPU to accelerate AI and ML workloads more efficiently than previous generations of Tensor Cores.
• Sparse matrix support: Third-generation Tensor Cores can also accelerate sparse matrix operations, which are common in many AI and ML workloads. This allows the A100 GPU to handle these types of workloads more efficiently, improving overall performance.
• Mixed-precision support: Third-generation Tensor Cores can perform calculations in mixed precision, which is a combination of single-precision (32-bit) and half-precision (16-bit) floating-point formats. This allows the A100 GPU to perform calculations faster and with higher accuracy than single-precision alone.

Fine-Grained Structured Sparsity

Fine-grained structured sparsity works by identifying and eliminating small groups of zero elements, or "microspikes," in a matrix. These microspikes can be difficult to identify and eliminate using traditional methods, but the A100 GPU's Tensor Cores are able to identify and eliminate them efficiently. This can significantly reduce the amount of data that needs to be processed by the GPU, improving the overall performance of AI and ML workloads.

Fine-grained structured sparsity is particularly useful for workloads that involve large, sparse matrices, such as those used in natural language processing (NLP) and recommendation systems. By eliminating microspikes, the A100 GPU can process these types of workloads more efficiently, improving the speed and performance of AI and ML tasks.

In the NVIDIA DGX Station A100 system architecture, fine-grained structured sparsity is used to improve the performance of AI and ML workloads by reducing the amount of unnecessary data that is processed by the GPU. It works in conjunction with the other components in the system, such as the CPU and memory, to provide fast and efficient performance for these types of workloads.

Multi-Instance GPU (MIG)

Multi-Instance GPU (MIG) is a feature of the NVIDIA A100 Tensor Core GPU that allows the GPU to be partitioned into multiple smaller instances, each with its own memory and compute resources. MIG is designed to improve the efficiency and utilization of the GPU, making it easier to run multiple workloads concurrently on the same GPU.

Some key capabilities and benefits of MIG for GPU acceleration include:

• Improved GPU utilization: By partitioning the GPU into multiple smaller instances, MIG allows multiple workloads to be run concurrently on the same GPU, improving utilization and making better use of the GPU's resources.
• Improved performance: MIG can improve the performance of GPU-accelerated workloads by allowing them to run concurrently on the same GPU, rather than having to wait for other workloads to complete.
• Better resource management: MIG allows administrators to allocate specific amounts of memory and compute resources to different workloads, allowing for better resource management and control.
  

Examples of how MIG can be used in different deployment scenarios include:

• Cloud environments: MIG can be used in cloud environments to allow multiple tenants to share the same GPU, improving utilization and reducing costs.
• HPC clusters: MIG can be used in HPC clusters to allow multiple users to share the same GPU, improving utilization and reducing the number of GPUs that are required.
• Data centers: MIG can be used in data centers to allow multiple servers to share the same GPU, improving utilization and reducing the number of GPUs that are required.

Improved Cooling System

To ensure that the DGX Station A100 can operate at optimal performance levels, it is equipped with an efficient cooling system. One of the key features of the cooling system in the DGX Station A100 is its ability to work quietly.

The system is designed to minimize noise levels, making it well-suited for use in a variety of environments, including offices and labs. The cooling system uses a combination of air cooling and water cooling to keep the GPUs and other components running at optimal temperatures, even under heavy workloads.

To further improve the efficiency of the cooling system, the DGX Station A100 is equipped with a number of advanced features, such as dynamic fan control and temperature-based speed control. These features allow the system to adjust its cooling performance in real-time, based on the workload and the ambient temperature, helping to ensure that the DGX Station A100 operates at optimal performance levels while minimizing noise levels.

Server-Class CPU

DGX Station A100 comes with the AMD Epyc 7742 enterprise-class server processor, which is based on the Zen 2 micro architecture. It employs the latest TSMC 7nm manufacturing process to offer the highest performance for AI workloads and HPC.

The DGX Station A100 system provides one of these CPUs for storage management, boot, and deep learning framework coordination and scheduling. It runs at a maximum of 3.4GHz and includes 64 cores using 2 threads per core.

The CPU offers extensive bandwidth and memory capacity, featuring 8 memory channels for an aggregate of 204.8 GB/s of memory bandwidth. Memory capacity on DGX Station A100 is 512GB standard including 8 DIMM slots populated with 64GB DDR4-3200 ECC RDIMM memory.

The AMD Epyc 7742 processor provides 128 PCIe Gen4 links for I/O to provide maximum bandwidth for high speed connectivity to GPUs and various IO devices. Each DGX Station A100 system includes one 7.68 TB PCIe gen4 NVMe U.2 cache SSDs, and 1.92 TB NVMe M.2 boot OS SSDs.

Added Flexibility with Remote Management (BMC)

The NVIDIA DGX Station A100 is equipped with a baseboard management controller (BMC), which is a specialized microcontroller that is responsible for monitoring the system's hardware and controlling its power state. The BMC is designed to provide added flexibility and remote management capabilities, allowing users to remotely monitor and control the DGX Station A100 from a remote location.

Some key features of the BMC in the DGX Station A100 include:

• Remote monitoring: The BMC allows users to remotely monitor the status of the DGX Station A100, including the temperature, power usage, and other key metrics.
• Remote power control: The BMC allows users to remotely power on or power off the DGX Station A100, as well as reset the system or shut it down in the event of an issue.
• Firmware updates: The BMC allows users to remotely update the firmware on the DGX Station A100, ensuring that the system is always running the latest version.

GPU Virtualization with Run.AI

Run:AI automates resource management and workload orchestration for machine learning infrastructure. With Run:AI, you can automatically run as many compute intensive experiments as needed on NVIDIA A100 and other data center grade GPUs.

Here are some of the capabilities you gain when using Run:AI:

• Advanced visibility—create an efficient pipeline of resource sharing by pooling GPU compute resources.
• No more bottlenecks—you can set up guaranteed quotas of GPU resources, to avoid bottlenecks and optimize billing.
• A higher level of control—Run:AI enables you to dynamically change resource allocation, ensuring each job gets the resources it needs at any given time.
  

Run:AI simplifies machine learning infrastructure pipelines, helping data scientists accelerate their productivity and the quality of their models.

Learn more about the Run:ai GPU virtualization platform.