Automate Build and Deploy

Use this option to streamline the build and deployment workflows. It keeps your model accurate by automatically re-training and deploying based on a cron expression, defined time interval, or metric base triggers.

You can also define a deployment conditions to verify that the new build passes acceptance criteria within the desired parameters before replacing a currently deployed model.

Automation Example

⚠️

Warning - Before Setting Up Automation

Prior to configuring automation, it is essential to have your model's code stored in a Git repository. It is recommended to confirm that all necessary Git repository access is correctly configured via CLI model builds. Ensure the JFrog ML model can successfully build from Git before proceeding with automation.

For additional details on building models from Git, refer to the Build Configurations page.

The automation fetches the model's code during the training process. When using a private repository, you must generate a Git access token and securely store the key in the Secret Manager.

from frogml.core.automations import Automation, ScheduledTrigger, FrogmlBuildDeploy,\
    BuildSpecifications, BuildMetric, ThresholdDirection, DeploymentSpecifications

test_automation = Automation(
    name="retrain_my_model",
    model_id="my-model-id",
    trigger=ScheduledTrigger(cron="0 0 * * 0"),
    action=FrogmlBuildDeploy(
        build_spec=BuildSpecifications(git_uri="https://github.com/org_id/repository_name.git#dir_1/dir_2",
                                       git_access_token_secret="token_secret_name",
                                       git_branch="main",
                                       main_dir="main",
                                       tags=["prod"],
                                       env_vars=["key1=val1", "key2=val2", "key3=val3"]),
        deployment_condition=BuildMetric(metric_name="f1_score",
                                         direction=ThresholdDirection.ABOVE,
                                         threshold="0.65"),
        deployment_spec=DeploymentSpecifications(number_of_pods=1,
                                                 cpu_fraction=2.0,
                                                 memory="2Gi",
                                                 variation_name="B")
    )
)

📘

Note - Scheduler Timezone

The default timezone for the cron scheduler is UTC.

Build & Deploy Configuration

The FrogmlBuildDeploy action has three configuration parameters:

1. build_spec defines the location of the model code that we will build in the JFrog ML platform.
2. deployment_condition defines the metrics used to determine when to deploy the model after the training.
3. deployment_spec specifies the runtime environment parameters for model deployment.

⚠️

Warning

Metrics used to trigger build or deploy automations must be logged during the model build phase.

BuildSpecifications

To configure the automation build specification, link to the git repository.

Note that the link consists of two parts delimited by hashtag #:

• The repository URL
• The path within the repository

For example, when using this link: https://github.com/org_id/repository_name.git#dir_1/dir2 , the platform will clone the https://github.com/org_id/repository_name.git repository and change the working directory to dir_1/dir_2 before starting the build.

In this example, dir_1/dir_2 should be the directory containing the main and tests folders.

Using Private Repositories

When using private repositories, the access token or private key must also be specified.

As the JFrog ML platform does not allow the usage of plain text token, the access tokens must be stored in the Secret Manager, and only the secret name should be specified.

If the default folder structure (in which main is the models folder) is not used, you must also specify the git branch and the directory containing the ML model.

Custom Resources

In the build specification, you can control the number of CPUs, amount of memory or use GPUs Instance Sizes

Defining CPU resources:

resources=CpuResources(cpu_fraction=2, memory="2Gi"))

Defining GPU resources:

resources=GpuResources(gpu_type="NVIDIA_K80", gpu_amount=1)

Alternatively, you can specify the instance type instead of fractions of resources. For example:

resources=ClientResources(instance='gpu.a10.8xl') #GPU

#OR

resources=ClientResources(instance='medium')  #CPU

You can specify the IAM role used in production (assumed_iam_role) or a custom docker image (base_image).

Environment Variables

Additionally, you can specify the environment variables to configure in the build environment. These should be specified with the env_vars field (list), and the value as the following:

key=value.

The model's code must log the metric that describes the model's performance. Use the metric in the deployment condition. If you don't know how to do it, look at our Logging and Monitoring Guide.

Disable Push Image

If you do not want the final build saved to the docker repository, disable the push image phase. You can do that by adding push_image=False to the BuildSpecification.

BuildMetric

During the build process, it is common to log metrics such as accuracy, F1 score, or loss. When executing the automation, these logged values may be compared against a specified threshold.

For each metric, you can define whether the value should be above or below the threshold. Once this condition is met, the JFrog ML platform will proceed to deploy the model.

The BuildMetric object has three parameters:

1. metric_name: The metric name you logged during the build phase
2. direction: Shows the value below or above the threshold, where the valid values are ThresholdDirection.ABOVE, ThresholdDirection.BELOW
3. threshold: The threshold used for comparison

⚠️

Warning

The threshold must always be a string, where threshold="0.65" is a valid threshold and threshold=0.65 is invalid!

Dynamic Threshold

To use a dynamic threshold, use an SQL expression as the threshold value.

In this case, the JFrog ML platform runs the SQL query in JFrog ML Model Analytics and compares the model's metric with the threshold produced by the SQL query.

The query must return a single row containing only one column.

DeploymentSpecifications

After the model has been built, its performance compared with the threshold, and it has been concluded that the model is ready to be deployed, the platform uses the deployment specification to configure the model's runtime environment.

You can specify:

Defining Auto-Scaling

To define an auto-scaling policy for the deployment, use the following pattern:

auto_scale_config = AutoScalingConfig(min_replica_count=1,
                                          max_replica_count=10,
                                          polling_interval=30,
                                          cool_down_period=300,
                                          triggers=[
                                              AutoScalingPrometheusTrigger(
                                                  query_spec=AutoScaleQuerySpec(
                                                      aggregation_type="max",
                                                      metric_type="latency",
                                                      time_period=4),
                                                  threshold=60
                                              )
                                          ]
                                          )