Google Storage V2 Cluster Binary Provider

This is the setting used for Google Cloud Storage using the filestore the using Google native client. when configuring filestore sharding for an HA cluster.

This is the setting used for Google Cloud Storage using the filestore using the Google native client. when configuring filestore sharding for an HA cluster. It is based on the sharding and dynamic provider logic that synchronizes the cluster-file-system.

When using the cluster-google-storage-v2 template*,* data is temporarily stored on the file system of each node using the Eventual Binary Provider, and is then passed on to your Google storage for persistent storage.

Each node has its own local filestore (just like in the file-system binary provider) and is connected to all other cluster nodes via dynamically allocated Remote Binary Providers using the Sharding-Cluster Binary Provider.

📘
Note
JFrog recommends using the Google Storage Binary Provider Native Client template instead of the Google Storage V2 Cluster Binary Provider, as described here.

cluster-google-storage template configuration

Because you must configure the google-storage-v2 provider with parameters specific to your account (but can leave all other parameters with the recommended values), if you choose to use the cluster-google-storage-v2 template, your binarystore.xml configuration file should look like this.

<config version="2">
    <chain template="cluster-google-storage-v2"/>
    <provider id="google-storage-v2" type="google-storage-v2">
        <endpoint>commondatastorage.googleapis.com</endpoint>
        <bucketName><BUCKET NAME></bucketName>
    </provider>
</config>

What's in the cluster-google-storage template?

While you don't need to configure anything else in your binarystore.xml, this is what the cluster-google-storage-v2 template looks like under the hood.

<config version="2">
 <chain> <!-- template="cluster-google-storage-v2" -->
        <provider id="cache-fs-eventual-google-storage" type="cache-fs">
            <provider id="sharding-cluster-eventual-google-storage" type="sharding-cluster">
                <sub-provider id="eventual-cluster-google-storage" type="eventual-cluster">
                    <provider id="retry-google-storage" type="retry">
                        <provider id="google-storage-v2" type="google-storage-v2"/>
                    </provider>
                </sub-provider>
                <dynamic-provider id="remote-google-storage" type="remote"/>
            </provider>
        </provider>
    </chain>
 
        <provider id="cache-fs-eventual-google-storage" type="cache-fs">
            <provider id="sharding-cluster-eventual-google-storage" type="sharding-cluster">
                <sub-provider id="eventual-cluster-google-storage" type="eventual-cluster">
                    <provider id="retry-google-storage" type="retry">
                        <provider id="google-storage-v2" type="google-storage-v2"/>
                    </provider>
                </sub-provider>
                <dynamic-provider id="remote-google-storage" type="remote"/>
            </provider>
        </provider>
  
    <provider id="sharding-cluster-eventual-google-storage" type="sharding-cluster">
        <readBehavior>crossNetworkStrategy</readBehavior>
        <writeBehavior>crossNetworkStrategy</writeBehavior>
        <redundancy>2</redundancy>
        <property name="zones" value="local,remote"/>
    </provider>
 
    <provider id="remote-google-storage" type="remote">
        <zone>remote</zone>
    </provider>
 
    <provider id="eventual-cluster-google-storage" type="eventual-cluster">
        <zone>local</zone>
    </provider>
 
    <provider id="google-storage-v2" type="google-storage-v2">
        <endpoint>commondatastorage.googleapis.com</endpoint>
        <bucketName><BUCKET NAME></bucketName>
    </provider>
</config>

For details about the cache-fs provider, see Cached Filesystem Binary Provider.

For details about the eventual provider, see Eventual Binary Provider.

For details about the retry provider, see Retry Binary Provider.

For details about the remote binary provider, see Remote Binary Provider.

For details about the sharding-cluster, see Sharding-Cluster Binary Provider.

Updated 2 days ago