Unique GPU Abstraction Capabilities
Run:AI’s platform includes advanced tech features in the area of GPU virtualization and abstraction. Fractional GPUs, Thin GPU Provisioning, and Job Swapping can bring AI cluster utilization to almost 100%, ensuring no resources are sitting idle.
Using Fractions of GPUs
Run:AI’s GPU abstraction capabilities allow GPU resources to be shared without memory overflows or processing clashes. Using virtualized logical GPUs, with their own memory and computing space, containers can use and access GPU Fractions as if they were self-contained processors.
The solution is transparent, simple and portable; it requires no code changes or changes to the containers themselves.
Thin GPU Provisioning
Allocation and Utilization are Different
With Thin GPU Provisioning, whenever a running workload is not utilizing its allocated GPUs, those resources can be provisioned and allocated to a different workload. This innovation is similar to thin provisioning used in storage systems. Run:AI makes this technology available for AI workloads on GPU resources, allowing for optimal GPU utilization.
Data Scientists are now removed from the details of scheduling and provisioning, as the Run:AI platform abstracts that from their day-to-day.
Run:AI’s Job Swapping feature enables the platform to seamlessly swap workloads that have been allocated the same GPU resources based on pre-set priorities. Together, Job Swapping and Thin GPU Provisioning ensure that enough GPU resources are available for all researchers in an automated way.