Define Automations

In the JFrog ML platform, you can easily define either time-based or trigger-based automations for model retraining or batch model executions.

Configure Automations

⚠️
Warning
The automation name must be unique throughout your models within the JFrog ML environment.

To configure an automation, create an instance of the Automation class and configure it:

from frogml.core.automations import Automation, ScheduledTrigger, FrogmlBuildDeploy,\
    BuildSpecifications, BuildMetric, ThresholdDirection, DeploymentSpecifications

test_automation = Automation(
    name="retrain_my_model",
    model_id="my-model-id",
    trigger=ScheduledTrigger(cron="0 0 * * 0"),
    action=FrogmlBuildDeploy(
        build_spec=BuildSpecifications(git_uri="https://github.com/org_id/repository_name.git#dir_1/dir_2",
                                       git_access_token_secret="token_secret_name",
                                       git_branch="main",
                                       main_dir="main",
                                       tags=["prod"],
                                       env_vars=["key1=val1", "key2=val2", "key3=val3"]),
        deployment_condition=BuildMetric(metric_name="f1_score",
                                         direction=ThresholdDirection.ABOVE,
                                         threshold="0.65"),
        deployment_spec=DeploymentSpecifications(number_of_pods=1,
                                                 cpu_fraction=2.0,
                                                 memory="2Gi",
                                                 variation_name="B")
    )
)

📘
Note - Scheduler Timezone
The default timezone for the cron scheduler is UTC.

Select an Instance Type

You can specify the purchase_option parameter of the BuildSpecifications to select between on-demand and spot instances. Available values: spot and ondemand. For example:

... 
build_spec=BuildSpecifications(
  git_uri="https://github.com/org_id/repository_name.git#dir_1/dir_2",
  git_access_token_secret="token_secret_name",
  git_branch="main",
  main_dir="main",
  tags=["prod"],
  env_vars=["key1=val1", "key2=val2", "key3=val3"],
  purchase_option="ondemand"
),
 ...

spot is the default value if you don't specify anything.

How to Trigger Automations

Automations run your workflow (for example evaluate, compare results to a threshold, deploy, or retrain) when a trigger fires. Triggers fall into two categories:

Schedule-based Triggers

Schedule-based automations run on a named interval or a cron expression.

For an interval, the configuration looks like this:

from frogml.core.automations import ScheduledTrigger

# Valid values: Daily, Weekly, Hourly
ScheduledTrigger(interval="Daily")

Metric-based Triggers

Use MetricBasedTrigger when you want to retrain (or otherwise act) because production performance—latency, errors, drift proxies, volume, or any metric you define from live usage—has crossed a threshold, instead of on a fixed schedule.

Provide a SQL query that returns a single numeric metric value from JFrog ML Model Analytics (the operational data for your deployed models). The trigger compares that value to your threshold using ThresholdDirection.

from frogml.core.automations import MetricBasedTrigger, ThresholdDirection, SqlMetric

MetricBasedTrigger(
    name='metric_name',
    metric=SqlMetric(sql_query='SQL_QUERY'),
    direction=ThresholdDirection.ABOVE,
    threshold="0.7"
)

Notifications

You can configure notifications when an automation succeeds or fails, using either Slack Webhooks or a custom webhook.

Slack Webhook

This option sends a Slack message when the automation finishes. The message includes the execution time, the model, the automation name, and the final status of the automation.

from frogml.core.automations import (
    SlackNotification,
    ScheduledTrigger,
    FrogmlBuildDeploy,
    BuildSpecifications,
    BuildMetric,
    DeploymentSpecifications,
    Automation,
)

test_automation = Automation(
    name="my_automation",
    model_id="my_model",
    trigger=ScheduledTrigger(cron="0 0 * * 0"),
    action=FrogmlBuildDeploy(
        build_spec=BuildSpecifications(...),
        deployment_condition=BuildMetric(...),
        deployment_spec=DeploymentSpecifications(...),
    ),
    on_error=SlackNotification(
        webhook="https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX",
    ),
    on_success=SlackNotification(
        webhook="https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX",
    ),
)

📘
Note
You can define alerting notifications for on_error and on_success triggers. on_success runs after a successful build when the deployment threshold was not met. It also runs after a successful build and deploy when the threshold was met.

Custom Webhook

You can also send notifications to a custom webhook with the following configuration:

from frogml.core.automations import CustomWebhook

on_success=CustomWebhook(
    url="http://my.api/endpoint/v1",
    http_method="GET", # defaults to "GET"
    data={"a1": "b1", "a2": "b2"},
    headers={"content-type": "application/json"}
)

This configuration calls the API (defined by the url) with the following parameters:

Execution time (execution_date)
Model ID (model_id)
Automation name (automation)
Execution Status (status)
Build ID (build_id)

The request also includes any fields you define in data.

If the HTTP method defined is GET, the data field plus the JFrog ML parameters above are sent as request parameters. For any other methods, they are sent in the request body.

Register the Automation

After you configure the automation and set up notifications, register it using the JFrog ML CLI:

frogml automations register -p .

In this command, -p is the path to the directory that contains your automation definitions. This example uses the current working directory.