Batch Deployments

This deployment type enables you to run batch inference executions in the system, and handle data files from an online cloud storage provider.

Deployment Configuration

Parameter	Description	Default Value
Model ID [Required]	The Model ID, as displayed on the model header.
Build ID [Required]	The JFrog ML-assigned build ID.
Initial number of pods	The number of k8s pods to be used by the deployment. Each pod handles one or more files/tasks.	1
CPU fraction	The CPU fraction allocated to each pod. The CPU resource is measured in CPU units. One CPU, in JFrog ML, is equivalent to: 1 AWS vCPU 1 GCP Core 1 Azure vCore 1 Hyperthread on a bare-metal Intel processor with Hyperthreading	2
Memory	The RAM memory (in MB) to allocate to each pod.	512
IAM role ARN	The user-provided AWS custom IAM role.	None
GPU Type	The GPU Type to use in the model deployment. Supported options are, NVIDIA K80, NVIDIA Tesla V100, NVIDIA T4 and NVIDIA A10.	None
GPU Amount	The number of GPUs available for the model deployment. Varies based on the selected GPU type.	Based on GPU Type
Purchase Option	Choose between `on-demand` or `spot` instances for the batch executions.	None (spot)
Service Account Key Secret Name	The service account key secret name to reach Google cloud services.	None

Batch Deployment from the UI

▶To deploy a batch model from the UI:

In the JFrog Platform, select AI/ML > Registry and select a custom model to deploy.
Select the Builds tab. Select a build to deploy and click the adjacent Deploy button. The Deployment configuration pane appears.
Select Batch.
Select the Instance type and the number of replicas.
Click Deploy model.

Batch Deployment from the CLI

▶ To deploy a model in batch mode from the CLI:

Populate the following command template:

frogml models deploy batch \
    --model-id <model-id> \
    --build-id <build-id> \
    --pods <pods-count> \
    --cpus <cpus-fraction> \
    --memory <memory-size>

For example, for the model built in the Get Started section, the deployment command is:

frogml models deploy batch \
    --model-id churn_model \
    --build-id 7121b796-5027-11ec-b97c-367dda8b746f \
    --pods 4 \
    --cpus 3 \
    --memory 1024