Autoscaling and Compute Resources

Autoscaling

Autoscaling allows Rhino to run Code Runs on additional compute resources when the default Rhino Client capacity is not sufficient. This capability is designed to support resource‑intensive, "bursty"(processing tasks occur in short, intense or irregular spurts), or GPU‑based workloads while improving isolation, reliability, and cost efficiency.

With autoscaling enabled, Rhino provisions temporary cloud VMs on demand, executes workloads on those VMs, and automatically terminates them when execution completes. This can help you make more efficient use of your computing resources and save on costs.

NOTE: This feature is only available if you are running on the Google Cloud Platform (GCP).

For NVFlare code objects and code runs, you can adjust autoscaling methods for both the NVFlare Server and the NVFlare Client. For the NVFlare Server, you can only specify the CPU and RAM.

---

Compute Options

There are 3 ways to specify compute for a code object run:

1

Use Client Resources

Uses the available client settings and resources.

How To:

1. Select the Use Client Resources button. It should be selected by default.

2. When complete, select Create New Code Object or start a Code Run.

2

Set Minimum

Allows you to set the minimum number of resources that must be available. If you choose the Set Minimum option, you can indicate the minimum number of CPUs, amount of RAM, the disk size (VRAM), as well as the type and number of GPUs. You can adjust these settings when you create a code object and also when you run your code.

Adjustments do not persist and will need to be reset with each run.

How To:

1. In the Required Compute Resources section, select Set Minimum.

2. Next, make changes to the settings as needed. The settings are explained below.

Setting	Description	Usage Note
CPU MIN	The minimum number of CPU cores on which your code will be run.	If the CPU MIN is too low, you might see slow code execution or throttling.
RAM MIN	The minimum amount of RAM (main memory) that you want to allocate to running your code.	If this is too low, you might see an out-of-memory crash or slowness in execution due to disk swapping.
DISK SIZE	Amount of storage allocated to the instance.	Increasing this setting increases capacity (not necessarily speed)
GPU TYPE	The model of GPU hardware. The number of GPUs indicate the number of GPUs attached to the instance.	If your code isn’t written to use multiple GPUs, extra GPUs might not be used. Note that GPU availability depends on the selected cloud region, Project‑level GPU quotas, and the current capacity at the cloud provider.

3

Use VM Pool

Allows you to select a Pre-allocated Virtual Machine (VM) Pool. Pre-allocated VM Pools allow you to indicate which dedicated compute resource you want to use for your code run, ensuring guaranteed availability for high-demand workloads such as GPU-intensive AI training and inference. This feature provides availability of critical compute resources and offers predictable costs through fixed capacity reservations.

A VM Pool selection is part of the code object's configuration.

How To:

See Using VM Pools

---

Table Summary

In the table below, we review the tradeoffs of each option.

Option	Capacity Comes From	Startup	Cost Shape	Best For
Use Client Resources	The Rhino Client's own default capacity	Immediate	Included	Routine work that fits existing client capacity
Set Minimum	A temporary cloud VM provisioned on demand, then terminated	Waits for provisioning	Varies per run	Bursty or irregular workloads; experimentation where shape changes run to run
Use VM Pool	A reservation of VMs held ready in advance	Faster — the VM already exists	Fixed, from the reservation	Recurring, high-demand, or GPU-intensive workloads where availability matters