> For the complete documentation index, see [llms.txt](https://docs.rhinofcp.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rhinofcp.com/creating-and-running-code-objects/creating-and-running-nvflare-code-and-running-inference.md).

# Creating and Running NVFlare Code and Running Inference

{% hint style="info" %}
To create and run NVFlare code using the Rhino SDK, see [Creating a New NVFlare Code Object Using the Rhino SDK](/rhino-sdk/creating-a-new-nvflare-code-object-using-the-rhino-sdk.md) and [Running NVFlare Code using the Rhino SDK](/rhino-sdk/running-nvflare-code-using-the-rhino-sdk.md).
{% endhint %}

## What is an NVFlare Code Object?

An NVFlare code object packages your NVFlare-ready application as a container image and makes it runnable in Rhino FCP.

NVFlare, short for NVIDIA Federated Learning Application Runtime Environment, is a groundbreaking software solution designed to revolutionize the world of federated machine learning. Built as an open-source and versatile SDK, NVFlare empowers researchers, data scientists, and platform developers to seamlessly transition their machine learning and deep learning workflows into the federated paradigm.

Use an NVFlare code object when you want to:

* Train a model across multiple institutions.
* Keep source data behind each institution's firewall.
* Reuse existing ML code with minimal changes.

### Key Features and Components

The FCP includes a seamless integration of [NVIDIA's Federated Learning framework (NVFlare)](https://developer.nvidia.com/flare), enabling you to train machine learning models collaboratively across distributed health data sources. This approach allows healthcare organizations to pool insights without compromising the security of sensitive data.

NVFlare's strength lies in its comprehensive features and components that enable efficient and secure federated learning:

* **Adapt Existing Workflows:** NVFlare allows you to adapt traditional machine learning and deep learning workflows, such as PyTorch, TensorFlow, Scikit-learn, XGBoost, and more, into a federated paradigm.
* **Componentized Architecture:** Built upon a componentized architecture, NVFlare provides flexibility from research and simulation to real-world deployment.
* **Federated Learning Algorithms:** NVFlare integrates key federated algorithms like FedAvg, FedProx, FedOpt, Scaffold, and Ditto.
* **Customizable Workflows:** With support for horizontal and vertical federated learning, NVFlare caters to a range of use cases.
* **Privacy Preservation:** NVFlare prioritizes data privacy through differential privacy and homomorphic encryption.
* **Secure Management:** NVFlare offers tools for secure provisioning, orchestration, and management.
* **Extensibility and Customization:** NVFlare's specification-based APIs allow researchers and developers to extend and customize the platform according to specific needs.

### Design Principles

NVFlare is built upon several core design principles:

* **Simplicity:** NVFlare adheres to a "less is more" approach, focusing on essential components and enabling others to customize and extend the platform.
* **Specification-Based:** Every component and API in NVFlare is specification-based, allowing alternative implementations to be developed by following the specifications.
* **Real-World Focus:** The platform is designed to address real-world challenges, with the ability to handle unexpected events and misbehaving code gracefully.
* **General Purpose:** NVFlare's general-purpose design ensures it's versatile and adaptable, catering to various federated computing use cases.
* **Client-Friendly:** NVFlare is built to be client-system-friendly, running with minimal environmental dependencies and integration interference.

### NVFlare Code Objects in Rhino FCP

An NVFlare Code Object is the way you package and run NVFlare-based workloads in Rhino FCP. It lets you execute federated training across a network of healthcare institutions, with each site contributing data insights without sharing raw data.

#### Key Aspects and Advantages

1. **Secure Distributed Training:** Each institution keeps sensitive patient information behind its own firewall while participating in Federated Training.
2. **NVIDIA GPU Acceleration:** NVFlare taps into the computational power of NVIDIA GPUs, which can reduce training time and improve model performance on large healthcare datasets.
3. **Versatility Across ML Frameworks:** NVFlare works with major machine learning frameworks such as PyTorch and TensorFlow, so you can adapt existing code for federated learning.

#### Unlocking Secure Federated Training

To use NVFlare Code Objects within Rhino FCP, follow these steps:

1. **Adaptation to NVFlare Framework:** Align your machine learning code with NVFlare's Federated Learning framework. See the [Rhino user resources GitHub repository](https://github.com/RhinoHealth/user-resources) for examples.
2. **Container Image Integration:** Build a container image that includes your adapted NVFlare code and required dependencies.
3. **ECR Repository Integration:** Push your container images to your workgroup's Elastic Container Registry (ECR) repository.
4. **Create and Collaborate:** In your project, create an NVFlare Code Object and specify your container image and configuration.
5. **Elevate Federated Training:** Launch Federated Training using NVFlare Code Objects. The FCP orchestrates the federated network and communication aspects for you.

#### Championing Secure Insights and Collaborative Progress

NVFlare Code Objects redefine the paradigm of Federated Training, emphasizing secure and collaborative healthcare insights. By utilizing the power of NVIDIA GPUs and preserving data privacy, NVFlare empowers healthcare organizations to collaborate, innovate, and drive transformative advancements in predictive modeling.

#### Summary

NVFlare Code Objects within Rhino FCP signify a powerful approach to Federated Training. They combine distributed health data, NVIDIA GPU acceleration, and federated learning framework compatibility to support secure and efficient model training.

### Where can I find more information about NVFlare?

* [The NVFlare Documentation Page](https://nvflare.readthedocs.io/en/main/index.html)
* [NVFlare's Official GitHub](https://github.com/NVIDIA/NVFlare)

## Creating an NVFlare Code Object

To create a new NVFlare Code object, do the following.

{% stepper %}
{% step %}

#### Go to the Code Objects page

Go to the main project's page and select your project.

Select **Code** from the menu on the left to open the **Code Objects** page.

![](/files/a4d6e63110746f2cd1bf91e14f72b752b699c0cb)
{% endstep %}

{% step %}

#### Open the Create New Code Object page

Select the **Create New Code Object** button to open the **Create New Code Object** page.

![](/files/31c26a981cd846ac73714a5e80d3c5ac7f101130)
{% endstep %}

{% step %}

#### Select the code object type.

Select **NVIDIA Flare** as the **Code Object Type**. The page changes to show the fields needed for this object type.

![](/files/6b537f18bf9a14eed523bde338b00329e36d5c1a) ![](/files/e8a8cbc2b2f795898c496147aaa2cf0b1262b8fa)
{% endstep %}

{% step %}

#### Fill in the fields

*Before you enter information in the fields (see table below), read the following about input and outputs*

*Code objects usually require that you know the number of inputs and outputs your code needs. Those inputs and outputs are provided as data schemas. For example, if your code splits one dataset into two (like a train/test split), you will have to create a code object with one input schema and two output schemas. For more detail into how to structure your code to run on FCP, see* [*Accessing your datasets in Code Objects*](/creating-and-running-code-objects/accessing-your-datasets-in-code-objects.md)*.*

*Multiple input and output datasets are supported in Python, Generalized, and Interactive Container Code Objects. While NVFlare Code Objects do not support multiple datasets, they can still be used as before for learning and inference across multiple sites.*

{% hint style="info" %}
**Pro-tip:** If you don't need this level of control, you can:

* **\[NEW]** set any input or output as optional, which will allow to provide or generate no datasets respectively
* **\[NEW]** set any input or output as a list, which will allow to select 1 or more datasets
* **\[NEW]** select "ANY" as input schema, which will allow code object re-use across any input dataset schema.
* let the system automatically determine the output schemas with [Auto-Generated Data Schemas](/data-schemas/auto-generated-data-schemas.md).
  {% endhint %}

Fill in the following fields.

| Field              | Description                                                                                                                                                                                                                             |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name               | Name of the code object.                                                                                                                                                                                                                |
| Description        | Description of the code object.                                                                                                                                                                                                         |
| Type               | Version of the NVIDIA FLARE code.                                                                                                                                                                                                       |
| Input Data Schema  | Data schema for the input datasets used by this code object. This selection determines which datasets are available when you run the code object.                                                                                       |
| Output Data Schema | Data schema for the output dataset produced by this code object. You can also select **\[Auto-generate Data Schema from Data]**. For more information, see [Auto-Generated Data Schemas](/data-schemas/auto-generated-data-schemas.md). |
| {% endstep %}      |                                                                                                                                                                                                                                         |

{% step %}

#### Select your container.

Next select your **container**. The Docker container you would like to execute when running this *Code*. This will either be a container you have created yourself and pushed to your ECR or a stock ECR image provided by Rhino. If you need a refresher on pushing containers to your ECR, see [Pushing Containers to the ECR](/getting-started/quick-start-guide/pushing-containers-to-the-ecr.md).

When you click on the drop-down to select your container, you will be presented with two different sections that contain various container images:

* **Workgroup Images:** These will be all the containers you have pushed to your workgroup's ECR. If you have not pushed anything to your workgroup's ECR, this will be blank.
* **Rhino Images:** These are a few container images that Rhino provides to users as utilities to help users. Currently, there are no NVFlare images that are provided by default from Rhino. For more information on the containers that Rhino provides, contact Rhino support.
  {% endstep %}

{% step %}

#### Set required compute resources

After you've completed your details, you can choose to set required compute resources. If you want to do this, go to the Setting Required Compute Resources (Auto-scaling) section on this page.
{% endstep %}

{% step %}

#### Create the code object

When complete, select the **Create New Code** button to create the new **Code Object**.

The code object appears on the **Code Object** page.
{% endstep %}
{% endstepper %}

### Setting Required Compute Resources (Autoscaling)

{% hint style="info" %}
NOTE: This feature is only available if you are running on the Google Cloud Platform (GCP). *For NVFlare code objects and code runs, you can adjust autoscaling methods for both the NVFlare Server and the NVFlare Client. For the NVFlare Server, you can only specify the CPU and RAM.*
{% endhint %}

Autoscaling allows Rhino to run Code Runs on additional compute resources when the default Rhino Client capacity is not sufficient. This capability is designed to support resource‑intensive, "bursty"(processing tasks occur in short, intense or irregular spurts), or GPU‑based workloads while improving isolation, reliability, and cost efficiency. With autoscaling enabled, Rhino provisions temporary cloud VMs on demand, executes workloads on those VMs, and automatically terminates them when execution completes. This can help you make more efficient use of your computing resources and save on costs.

In this section of the screen there are three options: Use Client Resources, Set Minimum, and Use VM Pool.

<figure><img src="/files/GFBA6RaRsa2CcbnjNlEN" alt=""><figcaption></figcaption></figure>

* **Use Client Resources** - Uses the available client settings and resources.
* **Set Minimum** - Allows you to set the minimum number of resources that must be available. If you choose the Set Minimum option, you can indicate the minimum number of CPUs, amount of RAM, the disk size (VRAM), as well as the type and number of GPUs. You can adjust these settings when you create a code object and also when you run your code. Note that adjustments do not persist and will need to be reset with each run.
* **Use VM Pool** - Allows you to select a Pre-allocated Virtual Machine Pool. Select a VM pool to indicate which dedicated compute resource you want to use for your code run, ensuring guaranteed availability for high-demand workloads such as GPU-intensive AI training and inference. This feature provides faster job startup times by using pre-warmed VMs and offers predictable costs through fixed capacity reservations.

Each option is detailed below.

#### *Use Client Resources*

To use client resources, complete the following steps.

1. Select the Use Client Resources button. It should be selected by default.
2. When complete, select Create New Code Object or start a Code Run.

#### *Set Minimum*

To set the minimum required compute resources, complete the following steps.

1. In the Required Compute Resources section, select Set Minimum.
2. Next, make changes to the settings as needed. Note that these setting changes do not persist the next time you run the code object or code run. The settings are explained below.

|           |                                                                                                     |                                                                                                                                                                                                                           |
| --------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Setting   | Description                                                                                         | Usage Notes                                                                                                                                                                                                               |
| CPU MIN   | The minimum number of CPU cores on which your code will be run.                                     | If the CPU MIN is too low, you might see slow code execution or throttling.                                                                                                                                               |
| RAM MIN   | The minimum amount of RAM (main memory) that you want to allocate to running your code.             | If this is too low, you might see an out-of-memory crash or slowness in execution due to disk swapping.                                                                                                                   |
| DISK SIZE | Amount of storage allocated to the instance.                                                        | Increasing this setting increases capacity (not necessarily speed)                                                                                                                                                        |
| GPU TYPE  | The model of GPU hardware. The number of GPUs indicate the number of GPUs attached to the instance. | If your code isn’t written to use multiple GPUs, extra GPUs might not be used. Note that GPU availability depends on the selected cloud region, Project‑level GPU quotas, and the current capacity at the cloud provider. |

#### *Use VM Pool*

To select a VM Pool, complete the following steps.

1. If you choose Use VM Pool, available pools are shown. For each pool you will see a name, as well as the current resources available. For each pool you will see the following information.

* Number of VMs (if any)
* Number of vCPUs per VM
* Amount of RAM per VM
* Disk size per VM
* Whether there are GPUs available, and how many

2. Select the pool of your choice.
3. When complete, select Create New Code Object or start a Code Run.

## View a Code Object's Configuration

To get an object's configuration, make sure you are on the page containing the original object you would like to view the configuration of. In the box where your original object is, there should be a row for each version. Navigate to the version you would like to view the configuration for and select the three-dot menu button, shown below:

![](/files/903f8d8b02de3537a785b7355db65dd33f4bf44c)

The menu button is on the right-hand side of the row, click it to view a new menu. The new menu should have an option to show that object's configuration, click on that to see the object's configuration.

For Code Objects and Code Runs, if you chose the Use VM Pool option, you will use the pool that was indicated during the Code Object or Code Run configurations.  Selecting this option shows the resources, including the number of VMs and vCPUs that will be allocated, as well as the amount of RAM, the size of the disk, the number of GPUs, the GPU type/mode&#x6C;*,* and the amount of VRAM.

## Creating a New Version of a NVFlare Code Object

To create a new version, complete the following steps.

1. Go to the page that has the original object you want to create a new version of.
2. In the upper right corner of the box where your original object is, select the **+ New Version** button.
3. A new window appears that allows you to create a new version of your object.

## Removing a Code Object

To remove a code object, complete the following steps.

{% stepper %}
{% step %}

#### Open the Code Objects page

Go to the page that lists your code objects.
{% endstep %}

{% step %}

#### Find the code object version

If needed, use the search fields at the top right.

* Use **Search Name** to search by code object name.
* Use **Search Description** to search by code object description.

Each code object appears with a row for each version. Find the version you want to delete.
{% endstep %}

{% step %}

#### Open the menu for the version

Select the three-dot menu on the right side of the version row, shown below.

![](/files/e30ab6c50f33a01039109108a719df8152c5cb07)
{% endstep %}

{% step %}

#### Remove the code object version

Select **Remove code object**.

Repeat these steps for each version you want to remove. When you delete the last version, the code object is fully deleted.
{% endstep %}
{% endstepper %}

## Running a NVFlare Code Object

To run NVFlare code, complete the following steps.

{% stepper %}
{% step %}

#### Go to the main project's page and select your project.

{% endstep %}

{% step %}

#### Select **Code** from the menu on the left to open the **Code Objects** page.

![](/files/29bc9aec9f274d39fe832de87fdd5fddc7d6d32c)
{% endstep %}

{% step %}

#### Select the Run button for the code object you'd like to run.

{% endstep %}

{% step %}

#### Fill in the following fields within the Code Run page.

{% hint style="info" %}
**Note:** These steps are the same for all NVIDIA FLARE Code Object versions 2.0, 2.2, 2.3, and 2.4.
{% endhint %}

<table><thead><tr><th width="236.08203125">Field</th><th>Description</th></tr></thead><tbody><tr><td>Training Datasets</td><td>One or many already imported Dataset to be used as input to your Code. If a Dataset happens to be a Collaborator's Dataset, the code will be run by your Collaborator's Rhino Client. In other words, the data never moves outside your collaborator's Rhino Client. If your Datasets all reside on your Rhino Client, make sure to check the below checkbox to simulate federated learning.</td></tr><tr><td>Simulated FL - One FL Client Per Training Dataset</td><td>Check this box to run simulated federated learning. When checked, your Rhino Client will spin up each Dataset as its own federation network, with each Dataset being its own "remote client," and perform federated training.</td></tr><tr><td>Validation Datasets (Optional)</td><td>An optional one or many Dataset that the newly trained federated model will be validated against.</td></tr><tr><td>Output Dataset Name Suffix</td><td>A suffix that is appended to the name of each input validation Dataset. This name will serve as the output validation Dataset that will be created during your validation run and then will be re-imported back into the system to display the results of your training validation.</td></tr><tr><td>Federated Server Config Override (Optional)</td><td>Copy and paste the contents of your <code>config_fed_server.json</code> file here. This will override <code>config_fed_server.json</code> in the config directory of the initial container image you pushed to your workspace's ECR. You can use this to dynamically set values for training, like hyper-parameters.</td></tr><tr><td>Federated Server Client Override (Optional)</td><td>Copy and paste the contents of your <code>config_fed_client.json</code> file here. This will override <code>config_fed_client.json</code> in the config directory of the initial container image you pushed to your workspace's ECR. You can use this to dynamically set values for training, like hyper-parameters.</td></tr><tr><td>Timeout (seconds)</td><td>The number of seconds that must elapse before a Code run is killed. This is to avoid zombie tasks that run perpetually within a Rhino client. The default is 1 hour.</td></tr></tbody></table>
{% endstep %}

{% step %}

#### Select the **Run** button to run your Code.

{% endstep %}
{% endstepper %}

## Reviewing Code Run Logs

Logs that are produced during a Code Run can be viewed by clicking on the Status link in the Code Runs view. If you need to search for a specific code run, use the text boxes located at the top right corner of the page. To search by name enter the full name or part of the name in the Search Name text box. To search by description, enter part of the code run's description in the Search Description text box.

![](/files/b9eac61eb5020042cd8f173ca83e2714a1f39575)

Clicking the Status link will open the Logs view for a particular run:

![](/files/913042c4d818bc668909f24d9376bea0c453f1c8)

Logs can be accessed while the Code Run is active and after the Code Run has completed. While the code is running, an auto-refresh option is available.

This Logs view provides the following information:

* General Info - information about the Code Run, including
  * Code Object name and type
  * Code Object version
  * Code Object description
  * Input and Output Datasets
  * Code Object configuration
  * Code Run configuration
* System Logs
  * Errors or warnings that occurred from the FCP side, e.g. Dataset import errors.
* FL server logs (for NVIDIA FLARE runs)
* FL client logs (for NVIDIA FLARE runs)
  * Each federated client's logs will appear separately
* Client-specific logs (for all other run types)

### Troubleshooting Code Runs

**Error Message that Usage Limit Has Been Exceeded**

If you get an error message indicating that the usage limit has been exceeded, your administrator has set usage limits on the number of input rows a workgroup can process per model within a certain timeframe. Getting this message means that the entire run has been cancelled. Please contact your administrator for more details and help with resolving the issue.

## Viewing a Code Run's Configuration

To get an object's configuration, make sure you are on the page containing the original object you would like to view the configuration of. In the box where your original object is, there should be a row for each version. Navigate to the version you would like to view the configuration for and select the three-dot menu button, shown below:

![](/files/903f8d8b02de3537a785b7355db65dd33f4bf44c)

The menu button is on the right-hand side of the row, click it to view a new menu. The new menu should have an option to show that object's configuration, click on that to see the object's configuration.

For Code Objects and Code Runs, if you chose the Use VM Pool option, you will use the pool that was indicated during the Code Object or Code Run configurations.  Selecting this option shows the resources, including the number of VMs and vCPUs that will be allocated, as well as the amount of RAM, the size of the disk, the number of GPUs, the GPU type/mode&#x6C;*,* and the amount of VRAM.

## Running Inference (NVFlare Code Runs)

{% hint style="info" %}
NOTE: Running Inference is available for NVFlare Code Runs only.
{% endhint %}

Once you’ve trained your model, you can use it to make inferences on other datasets. To do this, complete the following steps.

{% stepper %}
{% step %}
Select Code Runs from the menu to the left of the screen.
{% endstep %}

{% step %}
Right-click on the three-dot menu on the code you want to use to run inference, then select Run Inference.

![](/files/2767c77e3967046e4f8b37386b3997ba9f414e61)
{% endstep %}

{% step %}
The Run Inference screen opens.

![](/files/d15ffaff1ed0c3e18cc24684d9d4741212880f9e)
{% endstep %}

{% step %}
Select the validation dataset and indicate the output dataset name suffix if desired.
{% endstep %}

{% step %}
Under additional parameters, select the Model Parameters File.
{% endstep %}

{% step %}
If you need to adjust the timeout, which is the number of seconds that must elapse before a code run is automatically halted. This is an optional step.
{% endstep %}

{% step %}
Select Run Inference.
{% endstep %}
{% endstepper %}

## Deleting a Code Run

Follow these steps to delete a code run.

{% stepper %}
{% step %}

#### Open the Code Runs page

Go to the page that lists your code runs.
{% endstep %}

{% step %}

#### Find the code run

If needed, use the search fields at the top right.

* Use **Search Name** to search by code run name.
* Use **Search Description** to search by code run description.

Find the code run you want to delete.
{% endstep %}

{% step %}

#### Open the menu for the code run

Select the three-dot menu on the right side of the row, shown below.

![](/files/d6d3efbded53ff3765ccd54d866cfef8630e6c3c)
{% endstep %}

{% step %}

#### Remove the code run

Select **Remove code run**.
{% endstep %}
{% endstepper %}