Advanced File Operations with File Specs

You can perform complex file operations using multiple CLI commands. For example, you can upload different sets of files to different repositories. To simplify these operations, use the JFrog CLI download, upload, move, copy, and delete commands with the --spec option.

The --spec option replaces inline command arguments and options with a File Spec. Similarly, you can create and update release bundles using the --spec command option.

Each command uses a file specification array in JSON format. The schema for each command is described in the following sections. Note that if a command is issued with both inline options and a File Spec, the inline options override the corresponding settings in the File Spec.

File Spec Schemas

The following sections show the JSON schema for the different commands that support File Specs.

Copy and Move Commands Spec Schema

The file spec schema for the copy and move commands is as follows:

{
  "files": [
    {
      "pattern" or "aql": "[Mandatory]",
      "target": "[Mandatory]",
      "props": "[Optional]",
      "excludeProps": "[Optional]",
      "recursive": "[Optional, Default: 'true']",
      "flat": "[Optional, Default: 'false']",
      "exclusions": "[Optional, Applicable only when 'pattern' is specified]",
      "archiveEntries": "[Optional]",
      "build": "[Optional]",
      "bundle": "[Optional]",
      "validateSymlinks": "[Optional]",
      "sortBy": "[Optional]",
      "sortOrder": "[Optional, Default: 'asc']",
      "limit": "[Optional]",
      "offset": "[Optional]",
      "includeDeps": "[Optional, Default: 'false']",
      "excludeArtifacts": "[Optional, Default: 'false']",
      "project": "[Optional]"
    }
  ]
}

Download Command Spec Schema

The file spec schema for the download command is as follows:

{
  "files": [
    {
      "pattern" or "aql": "[Mandatory]",
      "target": "[Optional]",
      "props": "[Optional]",
      "excludeProps": "[Optional]",
      "recursive": "[Optional, Default: 'true']",
      "flat": "[Optional, Default: 'false']",
      "exclusions": "[Optional, Applicable only when 'pattern' is specified]",
      "archiveEntries": "[Optional]",
      "build": "[Optional]",
      "bundle": "[Optional]",
      "sortBy": "[Optional]",
      "sortOrder": "[Optional, Default: 'asc']",
      "limit": "[Optional]",
      "offset": "[Optional]",
      "explode": "[Optional, Default: 'false']",
      "bypass-archive-inspection": "[Optional, Default: 'false']",
      "validateSymlinks": "[Optional, Default: 'false']",
      "includeDirs": "[Optional, Default: 'false']",
      "includeDeps": "[Optional, Default: 'false']",
      "excludeArtifacts": "[Optional, Default: 'false']",
      "gpg-key": "[Optional]",
      "project": "[Optional]"
    }
  ]
}

Create and Update Release Bundle V1 Commands Spec Schema

The file spec schema for the create and update Release Bundle v1 commands is as follows:

{
  "files": [
    {
      "pattern" or "aql": "[Mandatory]",
      "pathMapping": "[Optional, Applicable only when 'aql' is specified]",
      "target": "[Optional]",
      "props": "[Optional]",
      "targetProps": "[Optional]",
      "excludeProps": "[Optional]",
      "recursive": "[Optional, Default: 'true']",
      "flat": "[Optional, Default: 'false']",
      "exclusions": "[Optional, Applicable only when 'pattern' is specified]",
      "archiveEntries": "[Optional]",
      "build": "[Optional]",
      "bundle": "[Optional]",
      "sortBy": "[Optional]",
      "sortOrder": "[Optional, Default: 'asc']",
      "limit": "[Optional]",
      "offset": "[Optional]",
      "project": "[Optional]"
    }
  ]
}

Upload Command Spec Schema

The file spec schema for the upload command is as follows:

{
  "files": [
    {
      "pattern": "[Mandatory]",
      "target": "[Mandatory]",
      "targetProps": "[Optional]",
      "recursive": "[Optional, Default: 'true']",
      "flat": "[Optional, Default: 'false']",
      "regexp": "[Optional, Default: 'false']",
      "ant": "[Optional, Default: 'false']",
      "archive": "[Optional, Must be: 'zip']",
      "exclusions": "[Optional]",
      "symlinks": "[Optional, Default: 'false']",
      "includeDirs": "[Optional, Default: 'false']",
      "explode": "[Optional, Default: 'false']",
      "project": "[Optional]"
    }
  ]
}

Search, Set-Props and Delete Commands Spec Schema

The file spec schema for the search, set-props, and delete commands is as follows:

{
  "files": [
    {
      "pattern" or "aql": "[Mandatory]",
      "props": "[Optional]",
      "excludeProps": "[Optional]",
      "recursive": "[Optional, Default: 'true']",
      "exclusions": "[Optional, Applicable only when 'pattern' is specified]",
      "archiveEntries": "[Optional]",
      "build": "[Optional]",
      "bundle": "[Optional]",
      "sortBy": "[Optional]",
      "sortOrder": "[Optional, Default: 'asc']",
      "limit": "[Optional]",
      "offset": "[Optional]",
      "includeDirs": "[Optional, Default: 'false']",
      "includeDeps": "[Optional, Default: 'false']",
      "excludeArtifacts": "[Optional, Default: 'false']",
      "project": "[Optional]"
    }
  ]
}

Build Add Dependencies Command Spec Schema

The file spec schema for the build-add-dependencies command is as follows:

{
  "files": [
    {
      "pattern": "[Mandatory]",
      "recursive": "[Optional, Default: 'true']",
      "exclusions": "[Optional]",
      "regexp": "[Optional, Default: 'false']",
      "from-rt": "[Optional, Default: 'false']"
    }
  ]
}

File Spec Examples

The following examples can help you get started with File Specs.

File Spec Example 1

Download all files from the all-my-frogs/ directory in the my-local-repo repository to the local froggy/ directory.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "target": "froggy/"
    }
  ]
}

File Spec Example 2

Download all files from all-my-frogs/ in my-local-repo, but only include files that are artifacts of build number 5 of the my-build build.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "target": "froggy/",
      "build": "my-build/5"
    }
  ]
}

File Spec Example 3

Download all files retrieved by the AQL query to the froggy/ directory.

{
  "files": [
    {
      "aql": {
        "items.find": {
          "repo": "my-local-repo",
          "$or": [
            {
              "$and": [
                {
                  "path": {
                    "$match": "."
                  },
                  "name": {
                    "$match": "a1.in"
                  }
                }
              ]
            },
            {
              "$and": [
                {
                  "path": {
                    "$match": "*"
                  },
                  "name": {
                    "$match": "a1.in"
                  }
                }
              ]
            }
          ]
        }
      },
      "target": "froggy/"
    }
  ]
}

File Spec Example 4

Upload all .zip files from the resources directory to the zip/ folder and all .tgz files to the tgz/ folder within the all-my-frogs repository. Tag .zip files with type=zip; status=ready and .tgz files with type=tgz; status=ready.

{
  "files": [
    {
      "pattern": "resources/*.zip",
      "target": "all-my-frogs/zip/",
      "targetProps": "type=zip;status=ready"
    },
    {
      "pattern": "resources/*.tgz",
      "target": "all-my-frogs/tgz/",
      "targetProps": "type=tgz;status=ready"
    }
  ]
}

File Spec Example 5

Upload all zip files located under the resources directory to the zip folder, under the all-my-frogs repository.

{
  "files": [
    {
      "pattern": "resources/*.zip",
      "target": "all-my-frogs/zip/"
    }
  ]
}

File Spec Example 6

Package all files located (including subdirectories) under the resources directory into a zip archive named archive.zip, and upload it into the root of the all-my-frogs repository.

{
  "files": [
    {
      "pattern": "resources/",
      "archive": "zip",
      "target": "all-my-frogs/"
    }
  ]
}

File Spec Example 7

Download all files located under the all-my-frogs directory in the my-local-repo repository except for files with the .txt extension and all files inside the all-my-frogs directory with the props. prefix.

Notice that the exclude patterns do not include the repository.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "exclusions": ["*.txt", "all-my-frog/props.*"]
    }
  ]
}

File Spec Example 8

Download the most recently uploaded file from the all-my-frogs/ directory in the my-local-repo repository.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "target": "all-my-frogs/files/",
      "sortBy": ["created"],
      "sortOrder": "desc",
      "limit": 1
    }
  ]
}

File Spec Example 9

Search for the three largest files located under the all-my-frogs directory in the my-local-repo repository. If there are files with the same size, sort them "internally" by creation date.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "sortBy": ["size", "created"],
      "sortOrder": "desc",
      "limit": 3
    }
  ]
}

File Spec Example 10

Download the second-latest file uploaded to the all-my-frogs directory in the my-local-repo repository.

{
  "files": [
    {
      "pattern": "my-local-repo/all-my-frogs/",
      "target": "all-my-frogs/files/",
      "sortBy": ["created"],
      "sortOrder": "desc",
      "limit": 1,
      "offset": 1
    }
  ]
}

File Spec Example 11

This example shows how to delete artifacts in Artifactory under specified path based on how old they are.

The following File Spec finds all the folders which match the following criteria:

They are under the my-repo repository.
They are inside a folder with a name that matches abc-*-xyz and is located at the root of the repository.
Their name matches ver*
They were created more than 7 days ago.

{
  "files": [
    {
      "aql": {
        "items.find": {
          "repo": "myrepo",
          "path": { "$match": "abc-*-xyz" },
          "name": { "$match": "ver*" },
          "type": "folder",
          "$or": [
            {
              "$and": [
                {
                  "created": { "$before": "7d" }
                }
              ]
            }
          ]
        }
      }
    }
  ]
}

File Spec Example 12

This example uses Using Placeholders. For each .tgz file in the source directory, create a corresponding folder in the target repository and upload the file there. For example, froggy.tgz is uploaded to my-local-repo/froggy/.

{
  "files": [
    {
      "pattern": "(*).tgz",
      "target": "my-local-repo/{1}/"
    }
  ]
}

File Spec Example 13

This example uses Using Placeholders. Upload all files whose name begins with frog to folder frogfiles in the target repository, but append its name with the text -up. For example, a file called froggy.tgz should be renamed froggy.tgz-up.

{
  "files": [
    {
      "pattern": "(frog*)",
      "target": "my-local-repo/frogfiles/{1}-up",
      "recursive": "false"
    }
  ]
}

File Spec Example 14

The following two examples lead to the exact same outcome.

The first one uses Using Placeholders, while the second one does not. Both examples download all files from the generic-local repository to be under the my/local/path/ local file-system path, while maintaining the original Artifactory folder hierarchy. Notice the different flat values in the two examples.

{
  "files": [
    {
      "pattern": "generic-local/{*}",
      "target": "my/local/path/{1}",
      "flat": "true"
    }
  ]
}

{
  "files": [
    {
      "pattern": "generic-local/",
      "target": "my/local/path/",
      "flat": "false"
    }
  ]
}

File Spec Example 15

This example creates a Release Bundle v1 and applies pathMapping to the artifact paths after distributing the Release Bundle.

All occurrences of the a1.in file are fetched and mapped to the froggy repository at the edges.

Fetch all artifacts retrieved by the AQL query.
Create the Release Bundle v1 with the artifacts and apply the path mappings at the Edge nodes after distribution.

The pathMapping option is provided, allowing users to control the destination of the Release Bundle artifacts at the edges.

To learn more, visit the Create Release Bundle v1.

{
  "files": [
    {
      "aql": {
        "items.find": {
          "repo": "my-local-repo",
          "$and": [
            {
              "name": {
                "$match": "a1.in"
              }
            },
            {
              "$or": [
                {
                  "path": {
                    "$match": "."
                  }
                },
                {
                  "path": {
                    "$match": "*"
                  }
                }
              ]
            }
          ]
        }
      },
      "pathMapping": {
        "input": "my-local-repo/(.*)",
        "output": "froggy/$1"
      }
    }
  ]
}

Schema Validation

JSON schemas allow you to annotate and validate JSON files. The JFrog File Spec schema is available in the JSON Schema Store catalog and at this link:

https://github.com/jfrog/jfrog-cli/blob/v2/schema/filespec-schema.json

Using JetBrains IDEs

JetBrains IDEs (IntelliJ IDEA, WebStorm, etc.) automatically apply the File Spec schema to files matching these patterns: **/filespecs/*.json, *filespec*.json, and *.filespec.

Using Visual Studio Code

To apply File Spec schema validation in VS Code, install the JFrog VS Code extension. Alternatively, add the following configuration to your settings.json file:

settings.json

"json.schemas": [
  {
    "fileMatch": ["**/filespecs/*.json", "*filespec*.json", "*.filespec"],
    "url": "https://raw.githubusercontent.com/jfrog/jfrog-cli/v2/schema/filespec-schema.json"
  }
]

Field Reference

Common Fields

The following fields are available across most file spec commands:

Field	Type	Default	Description
`pattern`	string	-	Specifies a local file system path or a path in Artifactory. Supports wildcards.
`aql`	object	-	An AQL query that specifies artifacts in Artifactory.
`target`	string	`./`	Specifies the target path for the operation.
`props`	string	-	List of `key=value` pairs separated by semicolons. Only artifacts with all specified properties are matched.
`excludeProps`	string	-	List of `key=value` pairs separated by semicolons. Artifacts with any of these properties are excluded.
`recursive`	string	`"true"`	If `"true"`, files are also collected from sub-folders.
`flat`	string	`"false"`	If `"true"`, artifacts are placed at the exact target path, ignoring source hierarchy.
`exclusions`	array	-	Array of patterns to exclude from the operation.
`archiveEntries`	string	-	If specified, only archive artifacts containing entries matching this pattern are matched.
`build`	string	-	If specified, only artifacts of the specified build are matched. Format: `build-name/build-number`.
`bundle`	string	-	If specified, only artifacts of the specified bundle are matched. Format: `bundle-name/bundle-version`.
`sortBy`	array	-	List of fields to sort by (e.g., `["created"]`, `["size", "name"]`).
`sortOrder`	string	`"asc"`	The order by which fields in `sortBy` should be sorted. Accepts `"asc"` or `"desc"`.
`limit`	integer	-	The maximum number of items to fetch. Usually used with `sortBy`.
`offset`	integer	-	The offset from which to fetch items (how many items to skip). Usually used with `sortBy`.
`project`	string	-	JFrog project key.

Upload-Specific Fields

Field	Type	Default	Description
`targetProps`	string	-	List of `key=value` pairs separated by semicolons. These properties are attached to uploaded artifacts.
`regexp`	string	`"false"`	If `"true"`, the pattern is interpreted as a regular expression.
`ant`	string	`"false"`	If `"true"`, the pattern is interpreted as an ANT pattern.
`archive`	string	-	Set to `"zip"` to pack and deploy files inside a ZIP archive.
`symlinks`	string	`"false"`	If `"true"`, preserves symbolic links structure in Artifactory.
`includeDirs`	string	`"false"`	If `"true"`, directories are also uploaded.
`explode`	string	`"false"`	If `"true"`, archives are extracted after upload.

Download-Specific Fields

Field	Type	Default	Description
`explode`	string	`"false"`	If `"true"`, archives are extracted after download. Supported formats: zip, tar, tar.gz, tgz.
`bypass-archive-inspection`	string	`"false"`	If `"true"`, bypasses security inspection of archives before extraction.
`validateSymlinks`	string	`"false"`	If `"true"`, validates that symlinks point to existing and unchanged files by comparing SHA1.
`includeDirs`	string	`"false"`	If `"true"`, directories are also downloaded.
`includeDeps`	string	`"false"`	If `"true"`, dependencies of the specified build are also matched. Requires `build` field.
`excludeArtifacts`	string	`"false"`	If `"true"`, build artifacts are excluded (only dependencies are matched). Requires `build` field.
`gpg-key`	string	-	Path to the public GPG key file for validating downloaded release bundles.

Release Bundle Fields

Field	Type	Default	Description
`pathMapping`	object	-	Defines source and target paths for artifacts after distribution. Requires `aql` field.
`pathMapping.input`	string	-	Input regex pattern for path mapping.
`pathMapping.output`	string	-	Output replacement pattern for path mapping.

Build Add Dependencies Fields

Field	Type	Default	Description
`from-rt`	string	`"false"`	If `"true"`, searches files in Artifactory instead of the local file system.