Trigger a Full Sync

Perform a Full Sync operation to synchronize all events between Federation members when regular synchronization is incomplete.

Who can perform this task: Platform Administrators

Prerequisites:

• Users running Artifactory 7.71.1+ should try auto-healing or manual Federation recovery before resorting to Full Sync

When to Use Full Sync

In certain cases, it might not be possible to synchronize all metadata events between Federation members. For example, if two members are disconnected for a long period (more than three days), old events might be deleted from the queue, making it impossible to copy those events after restoring the connection.

A Full Sync operation synchronizes all events in one JPD by copying any missing files to designated remote JPDs. However, Full Sync doesn't synchronize deleted events.

For example, if a file was deleted from JPD A but the event wasn't propagated to the remote site (JPD B) before the event was purged, running Full Sync on JPD B (where the file still exists) restores the file on JPD A. Therefore, if you need to synchronize the entire Federation, you must run Full Sync on all members.

Running a Full Sync

Full Sync is performed using the Federated Repository Full Sync REST API. You can synchronize specific Federated repositories or all the Federation repositories.

How Full Sync Works

When you initiate a Full Sync operation:

1. The query is sent to the remote member, which begins compiling a file list of the repository's contents.
2. The local JPD polls the remote JPD every 30 seconds to check whether the file list is ready. This polling method removes the need to keep the connection open for a long period.
3. When ready, the remote JPD sends its file list to the local JPD for comparison with the local file list.
4. The Full Sync operation begins, copying missing files.

flowchart LR
    trigger["Full Sync\ntriggered"] --> compile["Remote member compiles\nfile list"]
    compile --> poll["Local JPD polls\nevery 30 seconds"]
    poll --> ready{"File list\nready?"}
    ready -->|No| poll
    ready -->|Yes| compare["Compare local and\nremote file lists"]
    compare --> copy["Copy missing files"]
    copy --> complete["Member returns\nto Healthy"]

    classDef component fill:#41b375,stroke:#389b65,stroke-width:1px,color:#fff;
    classDef secondary fill:#ffffff,stroke:#41b375,stroke-width:1.5px,color:#41b375;
    class trigger,compile,poll,compare,copy,complete component;
    class ready secondary;

    linkStyle default stroke:#8e9aaf,stroke-width:1.5px;

When the Full Sync operation completes successfully, the member's state returns to Healthy (HEALTHY). If the operation fails, the member is flagged ERROR_OUT_OF_SYNC and may require another Full Sync attempt.

When using the Artifactory Federation Service (RTFS), a Full Sync operation that is interrupted can resume using a previously published file list instead of regenerating the list from the beginning. RTFS treats published file lists older than the configured time-to-live (TTL) as stale. For more information, see RTFS Properties for Full Sync File List Resume.

Perform Full Sync from a File

For very large Federated repositories, you can run a Full Sync operation using a pre-generated file list instead of compiling the remote file list over the network. This approach is available when the Artifactory Federation Service (RTFS) is enabled. Contact JFrog Support if you need access to this workflow.

The procedure is similar to legacy in-Artifactory Federation, except that the Full Sync file is placed on the local RTFS member filesystem (not on the Artifactory node). RTFS detects the file automatically and uses it as the remote member file list, skipping the remote fetch and polling steps.

Set Up the Source JPD

Perform the following steps on the JPD from which you want to trigger the Full Sync operation (the source or initiating member).

1. Exclude the repository from standard Full Sync by setting the following Artifactory system property to a comma-separated list of repository keys:

   artifactory.federated.full.sync.exclude.repos

   For example: fed1,fed2,fed3

   This prevents accidental triggers of the standard Federated Repository Full Sync REST API against the repository while you prepare the file-based workflow. The Federated Repository Force Full Sync REST API is used to run the operation in the final step.

2. Enable in-memory sorting for Full Sync operations. When RTFS is enabled, configure the RTFS property under rtfs.persistence.federationProperties in system.yaml or Helm values:

   YAML

   rtfs:
     persistence:
       federationProperties:
         rtfs.sync.in.memory.sort.enabled: "true"

   Restart Artifactory after updating the configuration. For legacy Federation (without RTFS), use artifactory.federated.repo.full.sync.enabled.inMemorySorter instead. See Bulk Mirroring Configuration.

Create the Full Sync File

On the target JPD (the remote Federation member whose metadata you want to gather), trigger the Create Full Sync File REST API:

POST /artifactory/api/fullsync/createFile

Sample request body:

{
  "repoKey": "fed2",
  "targetRepo": "example-repo-local",
  "path": "path/to"
}

The API creates the Full Sync file in the target repository and path you specify. The file name is <REPO_KEY>_fullsync_result.

Copy the file from the target repository to the initiating JPD for the next step.

Deploy the Full Sync File on the Local RTFS Member

On the source JPD (where you trigger Full Sync), copy the Full Sync file to the RTFS filesystem on the local RTFS member.

Directory:

$RTFS_HOME/var/data/rtfs/tmp/work/fullSync

File name:

<REMOTE_REPO_KEY>_<HOST>_fullsync_result

Example file name: fed2_remote-site.example.com_fullsync_result

Run the Full Sync File

On the source JPD, trigger the Force Full Sync REST API:

POST /artifactory/api/federation/forceFullSync/<REPO_KEY>

Replace <REPO_KEY> with the local Federated repository key. You can optionally pass a mirror query parameter to sync with a specific remote member only.

This API works like the standard Full Sync API, but RTFS uses the pre-placed file at $RTFS_HOME/var/data/rtfs/tmp/work/fullSync/<REMOTE_REPO_KEY>_<HOST>_fullsync_result as the remote member file list instead of fetching it from the remote JPD.

RTFS Properties for Full Sync File List Resume

When a Full Sync operation is interrupted (for example, by an RTFS restart), RTFS can resume using a previously published file list rather than compiling a new list from scratch. The rtfs.file.list.published.ttl.min property controls how long those published file lists remain valid:

• Default TTL: 30 minutes. Any file list published more than 30 minutes ago is treated as stale.
• Disable reuse: Set the value to 0 so RTFS always generates a new file list on resume instead of reusing a published one.
• Trade-off: A higher TTL speeds up resume after an interruption, but increases the chance RTFS resumes from an outdated file list if the repository already received newer metadata events. Consider a lower TTL, or 0, if artifacts changed significantly during the interruption.

The TTL drives two behaviors that share the same cutoff:

• Reuse vs. regenerate on resume: When resuming a Full Sync, RTFS reuses a published file list if it's still within the TTL. If the file list is stale, RTFS unpublishes it and regenerates a new file list.
• Cleanup of unused file lists: A scheduled cleanup job deletes published file-list task rows older than the TTL when the repository is not in use.

Configure this property under rtfs.persistence.federationProperties in system.yaml or Helm values:

rtfs:
  persistence:
    federationProperties:
      rtfs.file.list.published.ttl.min: "30"

Restart Artifactory after updating the configuration. See RTFS Configuration.

Use the Federation Comparison Tool for a Dry Run

The Federation Comparison tool performs a dry run of a Full Sync operation and returns the results in a JSON file. It compares the state of a Federated repository with one or more remote members to detect missing artifacts without actively synchronizing.

To use the Federation Comparison tool:

• Set the isDry query parameter in the Full Sync REST API to true:

  Bash

  curl -X POST "https://<ARTIFACTORY_INSTANCE>/artifactory/api/federation/fullSync/testDryFullSyncRepo?mirror=https://mirror-instance/artifactory/my-repo&isDry=true" \
       -H "Authorization: Bearer <ACCESS_TOKEN>" \
       -H "Content-Type: application/json"

  Where:

  • <ARTIFACTORY_INSTANCE>: The URL of your Artifactory instance
  • <ACCESS_TOKEN>: Your access token. For security, do not use a real token in examples. Use <ACCESS_TOKEN> as a placeholder

Sample response:

{
  "message": "Dry full sync completed successfully.",
  "resultPaths": [
    "jfrog-full-sync-info/comparison-results/testDryFullSyncRepo-mirror-comparison-2025-02-04.json"
  ]
}

Sample report contents:

{
  "results": [
    {
      "name": "testDryFullSyncFile_1.txt",
      "type": "file",
      "fullPath": "folder/testDryFullSyncFile_1.txt"
    },
    {
      "name": "testDryFullSyncFile_2.txt",
      "type": "file",
      "fullPath": "folder/testDryFullSyncFile_2.txt"
    }
  ]
}

The report is stored in the jfrog-full-sync-info system repository. The API response provides the path to the report.

System Properties for Full Sync File List Queries

The following Artifactory system properties define how to execute file list queries for Full Sync operations on Federated repositories.

The following properties are relevant only when artifactory.federated.full.sync.use.aql is set to false:

Frequently Asked Questions

This section provides answers to frequently asked questions.

Related Topics

• Recovery and Auto-Healing
• Bulk Mirroring Configuration
• System Properties Reference