Federated Repository Best Practices
Keep receiving-only members read-only and optimize Federation performance with recommendations for queues, workers, and timeouts.
Setting up Federated repositories on JPDs in a multisite environment impacts your organization's network, servers, and databases. It takes careful consideration and planning to configure a Federation that makes efficient use of available resources and provides timely and accurate synchronization across all Federation members.
The recommended approach is to build your repository Federations in stages, with thorough evaluations conducted at each stage to verify your infrastructure is configured properly to handle the demands of running repository Federations.
Keep Receiving-Only Members Read-Only
In a unidirectional (hub-and-spoke) Federation, treat the Federated repository on every receiving-only member as a read-only copy of the source. Perform all artifact changes on the sending member and let the content reach the receiving-only members through Federation.
Recommended, not enforced
Artifactory doesn't block local artifact changes on a receiving-only member. Uploads and deletions succeed on that JPD without any warning, which makes it easy to introduce drift unintentionally.
Recommendation: Don't upload, delete, move, or otherwise modify artifacts directly on a receiving-only member.
Why: A receiving-only member never sends events to the Federation, so local changes stay on that JPD only. This causes the following problems:
- The target is no longer a faithful copy of the source, which is the reason to use unidirectional sync in the first place.
- Content diverges between members, producing artifact and binary mismatches that are hard to detect.
- Federation monitoring still reports the member as synchronized, because Federation events continue to flow normally.
- Local changes aren't durable. A later Full Sync reconciles the member against the source, so locally added artifacts may be removed and locally deleted artifacts may be restored.
How to enforce it: Restrict write permissions on the Federated repository on the receiving-only JPD so that only Federation introduces changes. If a spoke must accept local uploads, use bidirectional Federation members with permissions, or keep a separate local repository on the spoke for locally produced artifacts.
Manual Migrations and Full Sync
Federated repositories rely on event-based synchronization. If you execute a manual migration process, for example moving data at the storage level or using methods that don't go through the Artifactory APIs, the necessary synchronization events aren't triggered.
Requirement: In such cases, you must execute a Full Sync on the impacted Federated repositories once the migration is completed to ensure all members are consistent.
Persisted Queue Age
This setting determines the amount of time that events remain in the queue before they are purged. The default value is three days. If there is a disconnection between two Federation members that lasts longer than three days, the older events are purged and can no longer be synchronized. You need to perform a Full Sync operation to re-establish the Federation.
If you increase this value to more than three days, it can greatly increase the size of your database table, which in turn increases costs and affects performance.
Property: artifactory.events.log.cleanup.age.of.entries.to.discard.days
Set to the same value for each Federation member. For example, in artifactory.system.properties:
artifactory.events.log.cleanup.age.of.entries.to.discard.days=3Number of Metadata Workers
Artifactory uses an internal mechanism (a "worker") to send metadata events in bulk via parallel threads from the source JPD to other Federation members. Each thread is managed by a worker, and each thread can handle the events for one Federated repository at a time. The default value is calculated based on the number of cores in the CPU.
The number of threads required is dependent on the number of repositories that are updated at the same time and the frequency of those updates. A sufficiently powerful server, for example a 64-core server, should provide sufficient power to manage 20-40 threads. See the Scaling and Sizing Guide for thread pool sizing recommendations by deployment tier.
Property: artifactory.persistentQueue.trigger.number.of.threads. Changing this property requires a restart.
Number of Binary Import Workers
Artifactory uses workers to import binary events from the source JPD to the target Federation members. The default value is 6. The number of workers required is dependent on the number of binaries the Federation is expected to handle, their relative size, and the capacity of your network.
Property: numberOfRemoteImporters, configured in binarystore.xml under the Federated provider section.
For more information about
binarystore.xml, see Binary Provider under Filestore Configuration. For the Federated Binary Provider on self-managed installations, see Federated Binary Provider.
Number of Thread Executors
Thread executors (pools), located in the target Federation member, enable the metadata events sent from the source Federation member to be uploaded in parallel while maintaining the correct order of the files.
Warning
The number of thread executors should always be equal to or greater than the number of metadata workers. Otherwise, a bottleneck results.
Property: artifactory.federated.mirror.events.bulk.max.pools
Number of Threads for Each Executor
This setting determines how many metadata events for a particular repository can be processed in parallel while preserving the correct order (for example, for Docker events this means uploading all layouts followed by all manifests). The default value is 4.
Property: artifactory.federated.mirror.events.bulk.threadsPerPool
Timeout Thresholds
You may need to increase the following timeout thresholds beyond their default values of 2 minutes if you expect your Federations to carry heavy loads.
artifactory.federated.mirror.events.bulk.executor.poll.timeout.minutesartifactory.federated.mirror.events.bulk.fullSync.executor.wait.timeout.minutes
Increase these values in proportion to the maximum expected load.
Increasing the Predefined Socket Timeout for Larger Repositories
From Artifactory version 7.38.17 for self-managed instances, you can increase the socket timeout when syncing large repositories between Federation members within a Federation.
Property: artifactory.mirror.http.client.socket.timeout.mili, set in artifactory.system.properties.
Network Infrastructure
The capabilities of your organization's network are of paramount importance to the optimal functioning of your Federation. The stronger the network connections between the members and the greater the bandwidth, the better the Federation is able to avoid latency when sending metadata events and binaries from site to site. See Network Architecture and Ports for port and firewall requirements.
Related Topics
Updated about 6 hours ago