RabbitMQ (Split) for Xray ( Xray Version >= 3.131.x )

This is a guide for installing Xray and RabbitMQ on separate nodes and connecting multiple Xray nodes to a RabbitMQ cluster (sequentially) to achieve high availability.

The setup supports native (rpm, deb), Linux archive, and docker-compose-based installation. This document uses three nodes for the application, RabbitMQ, and one Xray database node for demonstration purposes.

Why separate them?

• Improved Scalability with Dedicated Resources:

  Separating RabbitMQ allows for dedicated resource allocation (CPU, memory, and I/O), enabling it to scale independently from the application services.

• Reduced Resource Contention:

  When RabbitMQ and application services share the same resources, they compete for CPU, memory, and network bandwidth. Separating them eliminates this contention, ensuring more consistent and reliable performance.

• Simplified Troubleshooting: A clear separation between RabbitMQ and the application makes identifying and resolving issues easier, whether they stem from RabbitMQ or the application itself.

Before you start

Use a supported installer version:

Minimum hardware per node:

We recommend installing Rabbit MQ on at least three nodes in this deployment model. The table below lists the minimum hardware requirements for the servers. For a detailed description of the requirements, please refer to Xray System Requirements and Platform Support.

PostgreSQL

JFrog Xray requires PostgreSQL as its database. You could use a managed solution or a native installation. If you prefer the latter, we recommend using a separate standalone server for the database. For more details, refer to the following document: PostgreSQL for Xray.

Artifactory

Ensure Artifactory is operational on a separate, fully configured node.

The Big Picture

Whichever installer you use, the process is always the same two stages:

• Stage 1 — Build the RabbitMQ cluster (first node, then add more).
• Stage 2 — Build the Xray cluster and point it at RabbitMQ.

The only thing that changes between installers is the script name and the start command:

Detailed Installation Guide for all Installers

The below focuses on upgrade steps for different installer types using installv2.sh script

RPM / DEB

This process consists of two main stages: setting up the RabbitMQ cluster and configuring the Xray node cluster.

Stage 1: Setting Up the RabbitMQ Cluster

Adding First node to Form a RabbitMQ Cluster

• Starting Xray version 3.131.x, the installer script will optionally install RabbitMQ 4, enabled with Quorum queues.
• This stage focuses on configuring the RabbitMQ cluster. Since Xray relies on RabbitMQ for its operation, completing this step is crucial. Before proceeding, ensure that you are logged in as the root user.

1. Download the Xray RPM/DEB-based (tar.gz) from the JFrog Xray Downloads page.
2. Extract the tar.gz file
3. Run the installV2.sh script

./installV2.sh

4. Once you run the command, you will be prompted to select the products you wish to install. Choose the option RabbitMQ only.

5. Once you run the command, you will be prompted to confirm whether this node will join an existing cluster. Since this is a fresh installation on the first node, please type "N" to indicate no.

6. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.
7. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user

8. Start RabbitMQ service

systemctl start xray-rabbitmq.service

9. Access the UI through a browser to verify if the management plugin is enabled.

Adding Nodes to Form a RabbitMQ Cluster

Now that RabbitMQ is running on the first node, we need to install RabbitMQ on additional nodes and integrate them into the existing cluster. In this example, we will install RabbitMQ on two more nodes and join them to the cluster.

📘

NOTE: If you are installing an HA cluster, please use the same password for RabbitMQ across all nodes to successfully form the cluster.

1. Repeat steps 1 to 3 from the initial node setup to obtain the Xray files on the additional node.

2. Run installV2.sh file.

./installV2.sh

3. Once you execute the command, you will be prompted to confirm whether this node will join an existing RabbitMQ cluster. Since we already have an active RabbitMQ node, type Y and press Enter. Next, you will be asked to provide the name of the active RabbitMQ node. Here, you should input the hostname of the active RabbitMQ node.
4. You will be asked to provide the active node port and active node management port.
5. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.
6. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user
7. Start RabbitMQ service

systemctl start xray-rabbitmq.service

8. Repeat steps 1 through 6 to set up the additional RabbitMQ node and integrate it into the existing cluster.
9. Verify the RabbitMQ cluster status.
10. After installing and starting RabbitMQ on all required nodes, confirm that the cluster functions correctly and all nodes are properly joined.

❗️

Note: RabbitMQ Cluster Size and Quorum Queues When deploying a RabbitMQ cluster with more than 3 nodes, please be aware of the following behavior:

Quorum queues will be configured to grow on the first 3 nodes only. Any additional nodes beyond the first 3 will be ignored and will not participate as members of quorum queues.

Stage 2: Setting Up the Xray Node Cluster

Adding First node to Form a Xray Node Cluster

Once the RabbitMQ cluster is successfully configured, set up the Xray node cluster, which will integrate with the previously configured RabbitMQ cluster.

1. Download the Xray RPM/DEB-based (tar.gz) from the JFrog Xray Downloads page.
2. Extract the tar.gz file
3. Run the installV2.sh script

./installV2.sh

4. Once you run the command, you will be prompted to select the products you wish to install. Choose the option Xray only.
5. Once you run the command, you will be prompted to confirm whether this node will join an existing cluster. Since this is a fresh installation on the first node, please type "N" to indicate no.
6. The installer script will prompt the JFrog URL and the JoinKey of the Artifactory to join the JFrog Platform cluster. Find the join key in the JPD UI under the Administration module | Security | General | Connection Details. Enter your login password in the Current Password field and click Unlock.
7. You will be asked to enter the external RabbitMQ hostname, port, username and password. Enter the details from the RabbitMQ node installed above.
8. Start Xray Service

systemctl start xray.service

Adding Additional Nodes to Form an Xray Cluster:

1. Install Xray on a new node by repeating the initial Xray node setup steps. During the installation, you will be prompted to ask, "Are you adding an additional node to an existing product cluster? [y/N]?" At this point, select Y to confirm that this is an additional node for the X-ray cluster.
2. In the next step, you will be prompted to provide the master_key. You can retrieve this key from the first node

Linux Archive

This process consists of two main stages: setting up the RabbitMQ cluster and configuring the Xray node cluster.

Stage 1: Setting Up the RabbitMQ Cluster

Adding First Node to Form a RabbitMQ Cluster

Starting Xray version 3.131.x, installer script will install RabbitMQ 4 enabled with Quorum queues.

This stage focuses on configuring the RabbitMQ cluster. Since Xray relies on RabbitMQ for its operation, completing this step is crucial. Before proceeding, ensure that you are logged in as the root user.

1. Download the Xray Linux Archive (tar.gz) from the JFrog Xray Downloads page.
2. Extract the tar.gz file
3. OS user permissions for Linux archive When running Xray, the installation script creates a user called xray by default, which must have run and execute permissions on the installation directory.
   We recommend that you copy the Xray download file into a directory that gives run and execute permissions to all users such as /opt.

mkdir -p /opt/jfrog
cp -r jfrog-xray-<version>-linux /opt/jfrog/
cd /opt/jfrog
mv jfrog-xray-<version>-linux xray
cd xray/app/bin

📘

Note: From 3.107 onwards, JFrog Xray installers have organized files into designated subfolders. After extracting the tar.gz file, ensure you navigate to the correct directory

4. Run the installV2.sh script Check prerequisites for Xray in Linux Archive before running install script.

./installV2.sh

5. Once you run the command, you will be prompted to select the products you wish to install. Choose the option RabbitMQ only.
6. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.
7. Once you run the command, you will be prompted to confirm whether this node will join an existing cluster. Since this is a fresh installation on the first node, please type "N" to indicate no.
8. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user
9. Make sure to follow the instructions on the screen after the installation is completed regarding the below points
   1. Xray directory should be owned by the xray user eg : chown -R xray:xray /opt/jfrog/xray
   2. Switch to xray user, before you start the service eg: su xray
10. Start RabbitMQ service

xray/app/bin/xray.sh start|stop

11. Access the UI through a browser to verify if the management plugin is enabled.
12. (Optional) You can also install it as a service. Xray is packaged as an archive file and you can use the install script to install it as a service running under a custom user. Currently supported on Linux systems.

Adding Additional Nodes to Form a RabbitMQ Cluster

Now that RabbitMQ is running on the first node, we need to install RabbitMQ on additional nodes and integrate them into the existing cluster. In this example, we will install RabbitMQ on two more nodes and join them in the cluster.

1. Repeat steps 1 to 3 from the initial node setup to obtain the Xray files on the additional node.

2. Run installV2.sh file

./installV2.sh

1. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.

2. Once you execute the command, you will be prompted to confirm whether this node will join an existing RabbitMQ cluster. Since we already have an active RabbitMQ node, type Y and press Enter. Next, you will be asked to provide the name of the active RabbitMQ node. Here, you should input the hostname of the active RabbitMQ node.

3. You will be asked to provide the active node port and active node management port.

4. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user

5. Start RabbitMQ service

systemctl start xray-rabbitmq.service

6. Repeat steps 1 through 7 to set up the additional RabbitMQ node and integrate it into the existing cluster.
7. Verify the RabbitMQ cluster status.
8. After installing and starting RabbitMQ on all required nodes, confirm that the cluster functions correctly and all nodes are properly joined.

❗️

Note: RabbitMQ Cluster Size and Quorum Queues When deploying a RabbitMQ cluster with more than 3 nodes, please be aware of the following behavior:

Quorum queues will be configured to grow on the first 3 nodes only. Any additional nodes beyond the first 3 will be ignored and will not participate as members of quorum queues.

Stage 2: Setting Up the Xray Node Cluster

Adding First Node to Form an Xray Cluster

Once the RabbitMQ cluster is successfully configured, set up the Xray node cluster, which will integrate with the previously configured RabbitMQ cluster.

1. Download the Xray Linux (tar.gz) from the JFrog Xray Downloads page.
2. Extract the tar.gz file
3. OS user permissions for Linux archive When running Xray, the installation script creates a user called xray by default, which must have run and execute permissions on the installation directory. We recommend that you copy the Xray download file into a directory that gives run and execute permissions to all users such as /opt.

mkdir -p /opt/jfrog
cp -r jfrog-xray-<version>-linux /opt/jfrog/
cd /opt/jfrog
mv jfrog-xray-<version>-linux xray
cd xray/app/bin

📘

Note: From 3.107 onwards, JFrog Xray installers have organized files into designated subfolders. After extracting the tar.gz file, ensure you navigate to the correct directory

4. Run the installV2.sh script

./installV2.sh

5. Once you run the command, you will be prompted to select the products you wish to install. Choose the option Xray only.
6. Once you run the command, you will be prompted to confirm whether this node will join an existing cluster. Since this is a fresh installation on the first node, please type "N" to indicate no.
7. The installer script will prompt the JFrog URL and the JoinKey of the Artifactory to join the JFrog Platform cluster. Find the join key in the JPD UI under the Administration module | Security | General | Connection Details. Enter your login password in the Current Password field and click Unlock.
8. You will be asked to enter the external RabbitMQ hostname, port, username and password. Enter the details from the RabbitMQ node installed above.

9. Start Xray Service

xray/app/bin/xray.sh start|stop

Adding Additional Nodes to Form an Xray Cluster

1. Install Xray on a new node by repeating the initial Xray node setup steps. During the installation, you will be prompted to ask, "Are you adding an additional node to an existing product cluster? [y/N]?" At this point, select Y to confirm that this is an additional node for the X-ray cluster.
2. In the next step, you will be prompted to provide the master_key. You can retrieve this key from the first node.

Docker Compose

This process consists of two main stages: setting up the RabbitMQ cluster and configuring the Xray node cluster.

Stage 1: Setting Up the RabbitMQ Cluster

Adding First Node to Form an Xray Cluster

This stage focuses on configuring the RabbitMQ cluster. Since Xray relies on RabbitMQ for its operation, completing this step is crucial. Before proceeding, ensure that you are logged in as the root user.

1. Download the Xray docker compose-based installer(tar.gz) from the JFrog Xray Downloads page.
2. Extract the tar.gz file
3. Run configV2..sh file

./configV2.sh

1. Once you run the command, you will be prompted to select the products you wish to install. Choose the option RabbitMQ only.
2. Once you run the command, you will be prompted to confirm whether this node will join an existing cluster. Since this is a fresh installation on the first node, please type "N" to indicate no.
3. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.
4. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user
5. Start RabbitMQ service

docker-compose -p xray-rabbitmq -f docker-compose-rabbitmq.yaml up -d

6. Access the UI through a browser to verify if the management plugin is enabled.

Adding Additional Nodes to Form a RabbitMQ Cluster

Now that RabbitMQ is running on the first node, we need to install RabbitMQ on additional nodes and integrate them into the existing cluster. In this example, we will install RabbitMQ on two more nodes and join them in the cluster.

1. Repeat steps 1 to 3 from the initial node setup to obtain the Xray files on the additional node.

2. Run configV2.sh file

./configV2.sh

3. The script will prompt you for the selection of RabbitMQ version and Quorum Queue selection.
4. Once you execute the command, you will be prompted to confirm whether this node will join an existing RabbitMQ cluster. Since we already have an active RabbitMQ node, type Y and press Enter. Next, you will be asked to provide the name of the active RabbitMQ node. Here, you should input the hostname of the active RabbitMQ node.
5. You will be asked to provide the active node port and active node management port.
6. You will be asked to provide the custom username and password for RabbitMQ, default username guest will be used if custom username is not provided by the user
7. Start RabbitMQ service

docker-compose -p xray-rabbitmq -f docker-compose-rabbitmq.yaml up -d

8. Repeat steps 1 through 7 to set up the additional RabbitMQ node and integrate it into the existing cluster.
9. Verify the RabbitMQ cluster status.
10. After installing and starting RabbitMQ on all required nodes, confirm that the cluster functions correctly and all nodes are properly joined.

❗️

Note: RabbitMQ Cluster Size and Quorum Queues When deploying a RabbitMQ cluster with more than 3 nodes, please be aware of the following behavior:

Quorum queues will be configured to grow on the first 3 nodes only. Any additional nodes beyond the first 3 will be ignored and will not participate as members of quorum queues.