Migrate to JFrog Cloud

Transfer Artifactory Configuration and Files from Self-Hosted to JFrog Cloud

JFrog helps you migrate from a self-hosted JFrog Platform installation to JFrog Cloud. You can use JFrog CLI to transfer Artifactory configuration settings and binaries to JFrog Cloud, allowing you to transition seamlessly.

JFrog Cloud provides the same functionalities as a self-hosted JFrog Platform Deployment (JPD) without the overhead of managing databases and systems. If you are an existing JFrog self-hosted customer, you might want to move to JFrog Cloud to simplify operations. This solution allows you to replicate your self-hosted JPD to a JFrog Cloud JPD.

The Artifactory Transfer solution currently transfers the configuration and data of JFrog Artifactory only. Other products such as JFrog Xray and Distribution are not supported by this solution.

In this page, we refer to the source self-hosted instance as the source instance, and the target JFrog Cloud instance as the target instance.

Prerequisites and Requirements

Artifactory Version Support

The Artifactory Transfer solution is supported for:

• Artifactory 7.x
• Artifactory 6.23.21 and above

If your current version is not compatible, please consider upgrading your instance.

Supported OS Platforms

The transfer tool can transfer files and configuration from all types of operating systems, including Windows and container environments.

JFrog CLI Requirements

• JFrog CLI version 2.x or later
• Admin credentials for both source and target instances

Important: The data-transfer user plugin requires JFrog Artifactory Pro or Enterprise edition. Artifactory OSS does not support user plugins and cannot use the transfer-plugin-install or transfer-files commands.

Limitations

Keep the following limitations in mind before you start the migration process:

Transfer Phases

The transfer process includes two phases, which you must perform in the following order:

1. Configuration Transfer: Transfers configuration entities like users, permissions, and repositories from the source to the target instance.
2. File Transfer: Transfers the files (binaries) stored in the source instance repositories to the target instance repositories.

Notes:

• Files that are cached by remote repositories aren't transferred.
• The content of Artifactory's Trash Can isn't transferred.

You can perform both steps while the source instance is in use. No downtime on the source instance is required while the transfer is in progress.

Prerequisites Checklist

Before starting the transfer:

• If your source instance hosts files larger than 25 GB, contact JFrog Support to check whether this size limit can be increased.
• Ensure that you can log in to the UI of both the source and target instances with a user that has admin permissions.
• Ensure that the target instance license supports at least the same features as the source instance license.
• Run the file transfer pre-checks using --prechecks flag.
• Ensure that all remote and federated repositories on the source instance will have network access to their destination URLs once they are created on the target.
• Ensure that all replications configured on the source instance will have network access to their destination URLs.
• Ensure that you have a user who can log in to MyJFrog.
• Ensure that you can log in to the primary node of your source instance through a terminal.

Running the Transfer Process

Perform the following steps in the exact sequence provided. Do not run any of the commands in parallel.

Step 1: Enable Configuration Transfer on the Target Instance

By default, the target does not have the APIs required for the configuration transfer. Enabling the target instance for configuration transfer is done through MyJFrog. Once the configuration transfer is complete, you must disable the configuration transfer in MyJFrog as described in Step 4 below.

Warning:

• Enabling configuration transfer will trigger a shutdown of JFrog Xray, Distribution, Insights and Pipelines in the cloud and these services will therefore become unavailable. Once you disable the configuration transfer later on in the process, these services will be started up again.
• Enabling configuration transfer will scale down JFrog Artifactory, which will reduce its available resources. Once you disable the configuration transfer later on in the process, Artifactory will be scaled up again.

Steps:

1. Log in to MyJFrog.
2. Click Settings, then under the Transfer Artifactory Configuration from Self-Hosted to Cloud section, click on the acknowledgment checkbox. You cannot enable configuration transfer until you select the checkbox.
3. If you have an Enterprise+ subscription with more than one Artifactory instance, select the target instance from the drop-down menu.
4. Toggle Enable Configuration Transfer to enable the transfer. The process may take a few minutes to complete.
5. The configuration transfer is now enabled, and you can continue with the transfer process.

Step 2: Set Up the Source Instance for Pushing Files to the Target Instance

To set up the source instance, you must install the data-transfer user plugin in the primary node of the source instance.

Important: The data-transfer user plugin requires JFrog Artifactory Pro or Enterprise edition. Artifactory OSS does not support user plugins.

Steps:

1. Install JFrog CLI on the primary node machine of the source instance as described in Installing JFrog CLI on the Source Instance Machine.

2. Configure the connection details of the source Artifactory instance with your admin credentials:

   Bash

   jf c add source-server

3. Ensure that the JFROG_HOME environment variable is set and holds the value of the JFrog installation directory. It usually points to the /opt/jfrog directory.

If the source instance has internet access:

Download and install the data-transfer user plugin:

jf rt transfer-plugin-install source-server

If the source instance has no internet access:

1. Download the following two files from a machine that has internet access:

   • data-transfer.jar
   • dataTransfer.groovy

2. Create a new directory on the primary node machine and place the two files inside.

3. Install the plugin:

   Bash

   jf rt transfer-plugin-install source-server --dir "<plugin files dir>"

Manual Installation (Alternative):

1. Download data-transfer.jar and place it under $JFROG_HOME/artifactory/var/etc/artifactory/plugins/lib/. Create the lib directory if not present.
2. Download dataTransfer.groovy and place it under $JFROG_HOME/artifactory/var/etc/artifactory/plugins/.
3. Reload the plugin:
   Bash

   curl -u admin -X POST http://localhost:8082/artifactory/api/plugins/reload

Step 3: Transfer Configuration from the Source Instance to the Target Instance

Warning: The following process will wipe out the entire configuration of the target instance, and replace it with the configuration of the source instance. This includes repositories and users.

Steps:

1. Install JFrog CLI on the source instance machine.

2. Configure the connection details of the source Artifactory instance:

   Bash

   jf c add source-server

3. Configure the connection details of the target Artifactory server:

   Bash

   jf c add target-server

4. Run pre-checks to verify target URLs are accessible:

   Bash

   jf rt transfer-config source-server target-server --prechecks

5. Transfer the configuration:

   Bash

   jf rt transfer-config source-server target-server

   This command might take up to two minutes to run.

Notes:

• By default, the command will not transfer if the target instance isn't empty. Use --force to override.
• Use --include-repos and --exclude-repos to selectively transfer repositories.

The command output is divided into four phases:

========== Phase 1/4 - Preparations ==========
========== Phase 2/4 - Export configuration from the source Artifactory ==========
========== Phase 3/4 - Download and modify configuration ==========
========== Phase 4/4 - Import configuration to the target Artifactory ==========

Step 4: Disable Configuration Transfer on the Target Instance

Once the configuration transfer is successful, disable configuration transfer for security and performance reasons.

1. Login to MyJFrog
2. Under the Actions menu, choose Enable Configuration Transfer
3. Toggle Enable Configuration Transfer to off

Disabling the configuration transfer might take some time.

Step 5: Push the Files from the Source to the Target Instance

Running Pre-checks:

Before initiating the file transfer, run pre-checks to identify potential issues:

jf rt transfer-files source-server target-server --prechecks

This validates:

• Network connectivity between source and target
• Artifacts with properties exceeding 2.4K character limit

Initiating File Transfer:

jf rt transfer-files source-server target-server

This command may take a few days depending on system size and network speed.

Monitoring Progress:

If running in the background, view progress with:

jf rt transfer-files --status

Stopping Transfer:

jf rt transfer-files --stop

The process will continue from where it stopped when you re-run the command.

Step 6: Sync Configuration Between Source and Target

After file transfer completes, you may want to sync configuration if new entities were created during the transfer. Simply repeat Steps 1-3.

Command Reference

transfer-config

Copy full Artifactory configuration from source to target server.

Syntax:

jf rt transfer-config [command options] <source-server-id> <target-server-id>

Arguments:

Options:

Example:

# Transfer with specific repositories
jf rt transfer-config source-server target-server --include-repos="libs-*;apps-*"

# Force transfer to non-empty target
jf rt transfer-config source-server target-server --force

# Run pre-checks only
jf rt transfer-config source-server target-server --prechecks

transfer-config-merge

Merge projects and repositories from source to target without wiping existing configuration.

Syntax:

jf rt transfer-config-merge [command options] <source-server-id> <target-server-id>

Arguments:

Options:

Note: Users are NOT transferred during merge. Only repositories and projects are merged with the cloud instance.

Example:

# Merge specific project and repository
jf rt transfer-config-merge source-server target-server --include-repos="my-repo" --include-projects="my-project"

transfer-files

Transfer files from one Artifactory instance to another.

Syntax:

jf rt transfer-files [command options] <source-server-id> <target-server-id>

Arguments:

Options:

Examples:

# Transfer all files
jf rt transfer-files source-server target-server

# Transfer specific repositories
jf rt transfer-files source-server target-server --include-repos="libs-release;libs-snapshot"

# Transfer with file path filtering
jf rt transfer-files source-server target-server --include-files="com/mycompany/*"

# Check transfer status
jf rt transfer-files --status

# Stop running transfer
jf rt transfer-files --stop

# Run pre-checks only
jf rt transfer-files source-server target-server --prechecks

# Restart transfer from scratch
jf rt transfer-files source-server target-server --ignore-state

# Use with pre-copied filestore
jf rt transfer-files source-server target-server --filestore

transfer-plugin-install

Download and install the data-transfer user plugin on the primary node of Artifactory.

Important: Requires JFrog Artifactory Pro or Enterprise edition. Artifactory OSS does not support user plugins.

Syntax:

jf rt transfer-plugin-install <server-id> [command options]

Arguments:

Options:

Note: The --dir and --version flags are mutually exclusive. Use --dir to install from local files, or --version to download a specific version from the internet.

Examples:

# Install latest version (requires internet)
jf rt transfer-plugin-install source-server

# Install specific version
jf rt transfer-plugin-install source-server --version=1.2.0

# Install from local files (no internet required)
jf rt transfer-plugin-install source-server --dir="/path/to/plugin/files"

# Install to custom JFrog home
jf rt transfer-plugin-install source-server --home-dir="/custom/jfrog/home"

transfer-settings

Configure settings for the jf rt transfer-files command interactively.

Syntax:

jf rt transfer-settings

Description:

This interactive command allows you to configure file transfer settings, including:

• Number of working threads: Controls parallel threads for file transfer. Default is 8 threads. Increase for faster transfers (higher load) or decrease for slower transfers (lower load).

Settings are cached in ~/.jfrog/transfer/ and used for subsequent runs of transfer-files from the same machine.

Note: When transferring files in build-info repositories, JFrog CLI limits threads to 8 regardless of this setting to limit load on the target instance.

Example:

jf rt transfer-settings

How Does File Transfer Work?

File Transfer Phases

The jf rt transfer-files command pushes files from source to target as follows:

• Files are pushed for each repository, one by one in sequence.
• For each repository, the process includes three phases:

• If Phase 1 finished for a repository, subsequent runs only trigger Phase 2 and Phase 3.
• You can run transfer-files as many times as needed until ready to move traffic to target permanently.

Transfer State

JFrog CLI stores transfer state in ~/.jfrog/transfer/ directory. This allows:

• Resuming interrupted transfers
• Skipping already-transferred files
• Retrying failed files

To ignore stored state and restart from scratch:

jf rt transfer-files source-server target-server --ignore-state

Controlling File Transfer Speed

By default, transfer-files uses 8 working threads. This can be adjusted while transfer is running:

jf rt transfer-settings

Recommendations:

• Reduce threads for slower transfer speed and lower source instance load
• Increase threads gradually for faster transfers while monitoring load
• Settings are cached and reused for subsequent runs

Routing Traffic Through an HTTPS Proxy

If traffic from source must route through an HTTPS proxy:

1. Define proxy details in source instance UI (Managing Proxies)
2. Add --proxy-key option to transfer command:

jf rt transfer-files my-source my-target --proxy-key my-proxy-key

Manually Copying the Filestore

For instances with hundreds of terabytes, you may manually copy the entire filestore to JFrog Cloud storage to reduce transfer time. This requires coordination with your JFrog account manager.

After copying the filestore, run:

jf rt transfer-files source-server target-server --filestore

This transfers only metadata (paths, names, properties, statistics) and files created/modified after the copy.

Using Replication

To reduce Phase 2 time, configure Event-Based Push Replication for local repositories. Files created/updated on source are immediately replicated to target.

Replication can be configured at any time: before, during, or after file transfer.

Installing JFrog CLI on the Source Instance Machine

Install using JFrog CLI Installers:

curl -fL https://install-cli.jfrog.io | sh

For Docker containers:

1. Connect to host machine terminal
2. Download CLI:
   Bash

   curl -fL https://getcli.jfrog.io/v2-jf | sh

3. Copy to container:
   Bash

   docker cp jf <container-name>:/usr/bin/jf

4. Verify in container:
   Bash

   jf -v

Installing JFrog CLI on a Machine with Network Access to Both Instances

For transfer-files, use a machine with network access to both source and target URLs to spread transfer load across cluster nodes.

1. Install JFrog CLI:

   Bash

   curl -fL https://install-cli.jfrog.io | sh

2. Set proxy environment variable if needed (see Proxy Support documentation)

3. Configure source server:

   Bash

   jf c add source-server

4. Configure target server:

   Bash

   jf c add target-server

Transferring Projects and Repositories from Multiple Source Instances

To merge configuration from multiple sources to a single target while preserving existing configuration:

1. Ensure admin access tokens for both source and target (not username/password)
2. Install JFrog CLI with network access to both instances
3. Run merge command:
   Bash

   jf rt transfer-config-merge source-server target-server

Note: Only projects and repositories are transferred. Users are not transferred during merge.

Exceptional Cases

Transferring Files Larger Than 25GB

Check for large files:

curl -X POST <source-url>/artifactory/api/search/aql \
  -H "Content-Type: text/plain" \
  -d 'items.find({"name":{"$match":"*"}}).include("size","name","repo").sort({"$desc":["size"]}).limit(1)' \
  -u "<USERNAME>:<PASSWORD>"

If size exceeds 25000000000, contact [email protected] before transferring.

Troubleshooting

Temporary Directory Error on RHEL/CentOS

Error:

Error: Creating temp export directory: /export/jfrog-cli/tmp/...
500 : Failed to create backup dir: ...

Solution:

mkdir ~/tmp
sudo chown -R artifactory:artifactory ~/tmp
export JFROG_CLI_TEMP_DIR=~/tmp

File Count Differences

Differences between source and target file counts are expected due to cleanup policies triggered during deployment.

Validating Transfer Completion

JFrog CLI validates transfers automatically:

1. Traverses source repositories and pushes files
2. Logs errors with file details on failures
3. Provides summary of failed files at end
4. Stores failures in ~/.jfrog/transfer/ for retry

File Integrity

Source Artifactory stores checksums for every file. During transfer:

• Checksums sent as HTTP headers
• Target calculates and compares checksums
• Mismatches trigger re-transfer attempts

Frequently Asked Questions

Q: Can I stop and restart the transfer command?

A: Yes. Stop with CTRL+C or --stop flag. State is saved in ~/.jfrog/transfer/ and subsequent runs resume from where stopped.

Q: Why is the file count different after transfer?

A: Cleanup policies and other Artifactory behaviors can affect file counts. This is expected.

Q: How do I verify all files transferred successfully?

A: JFrog CLI validates during transfer. Review the error summary CSV at completion for any failures.

Quick Reference