Query Output and Modifiers

This page covers how to control what fields appear in query results, how to sort and paginate output, and the limitations that apply to output modifiers.

Specify Output Fields

Each query displays a default set of fields in the result set, however you have complete control this and may specify which fields to display using an optional include element in your query.

You can even specify to display fields from other entities related to your result set.

Display All Fields

Use:.include("*")

For example:

//Find all items, and display all the item fields
items.find().include("*")

Display Specific Fields

Each query displays a default set of fields in the output. Using the.include element you can override this default setting and specify any particular set of fields you want to receive in the output.

Use: .include(" ", " "...)

For example:

//Find all items, only display the "name" and "repo" fields
items.find().include("name", "repo")

You can also display specific fields from other entities associated with those returned by the query.

If you specify any field from the item domain, then this will override the default output setting, and only the item fields you expressly specified will be displayed.

If you only specify fields from the property or stat domains, then the output will display the default fields from the item domain, and in addition, the other fields you expressly specified from the property or stat domains.

For example:

//Find all items, and display the "name" and "repo" fields as well as the number of "downloads" from the corresponding "stat" entity
items.find().include("name", "repo", "stat.downloads")

//Find all items, and display the default item fields fields as well as the stat fields
items.find().include("stat")

//Find all items, and display the default item fields as well as the stat and the property fields
items.find().include("stat", "property")

//Find all items, and display the "name" and "repo" fields as well as the stat fields
items.find().include("name", "repo", "stat")
 
//Find all builds that generated items with an Apache license, and display the build fields as well as the item "name" fields. Click below to view the output of this query
builds.find({
                "module.artifact.item.@license":{"$match":"Apache*"}
            }
).include("module.artifact.item.name")

The output displays the default fields of the "build" domain, and the "name" field from the item domain. Fields from the module and artifact domains are not displayed since they were not specified in the include element.

{

"results" : [ {
  "build.created" : "2015-09-06T15:49:01.156+03:00",
  "build.created_by" : "admin",
  "build.name" : "maven+example",
  "build.number" : "313",
  "build.url" : "http://localhost:9595/jenkins/job/maven+example/313/",
  "modules" : [ {
    "artifacts" : [ {
      "items" : [ {
        "name" : "multi-3.0.0-20150906.124843-1.pom"
      } ]
    } ]
  } ]
},{
  "build.created" : "2015-09-06T15:54:40.726+03:00",
  "build.created_by" : "admin",
  "build.name" : "maven+example",
  "build.number" : "314",
  "build.url" : "http://localhost:9595/jenkins/job/maven+example/314/",
  "modules" : [ {
    "artifacts" : [ {
      "items" : [ {
        "name" : "multi-3.0.0-20150906.124843-1.pom"

      } ]
    } ]
  } ]
} ],
"range" : {
  "start_pos" : 0,
  "end_pos" : 2,
  "total" : 2
}
}

Filtering Properties by Key

The primary use of the .include element is to specify output fields to display in the result set.

This notion is applied in a similar way in regard to properties. Each item may be annotated with several (even many) properties. In many cases, you may only be interested in a specific subset of the properties, and only want to display those.

So the.include element can be used to filter out unwanted properties from the result, and only display (that is,include) those you are interested in. However, note that.include will also filter out items that do not have this property.

For example, to display all the properties annotating an item found.

//Find all items, and display the "name" and "repo" fields, as well as all properties associated with each item
items.find().include("name", "repo", "property.*")

However, if you are only interested in a specific property (for example, you want to know the version of each item returned), you can filter out all other properties and only include the property with the key you are interested in:

//Find all items, and display the "name" and "repo" fields, as well as the key and value of the "version" property of each item
items.find().include("name", "repo", "@version")

Sorting in AQL

AQL implements a default sort order, however, you can override the default and specify any other sort order using fields in your output by adding the.sort element to the end of your query as follows:

.sort(\{"<$asc | $desc>" : [" ", " ",... ]\})

Here are some examples:

 // Find all the jars in artifactory and sort them by repo and name
 items.find({"name" : {"$match":"*.jar"}}).sort({"$asc" : ["repo","name"]})

 // Find all the jars in artifactory and their properties, then sort them by repo and name
items.find({"name" : {"$match":"*.jar"}}).include("@").sort({"$asc" : ["repo","name"]})

Limitation

The.sort element has the following limitation:

• If your query has aninclude element, you can only specify fields from the primary domain in it.

For example, in the following query,.sort will not work because the primary domain is anitem, but theinclude element specifies that fields from theartifact,module, andbuild domains should be displayed:

items.find().include("artifact","artifact.module","artifact.module.build")

This means that if you search for an item and include the property you will not be able to sort by the property.

items.find({"repo":"example-repo-local"}).include("repo","path", "name", "created", "@build.number")

Display Limits and Pagination

.limit(N) sets the maximum number of results Artifactory returns.

// Find jars and sort them by repo and name, but return at most 100 results
items.find({"name" : {"$match":"*.jar"}}).sort({"$asc" : ["repo","name"]}).limit(100)

Return the Results You Can Read

From Artifactory 7.133.0, you can ask Artifactory to search further and still receive at most N results.

Add the scan-factor header to POST /artifactory/api/search/aql. Use an integer greater than 1. Artifactory multiplies your .limit(N) by that number when it searches, then stops after N results you can read.

For example, .limit(50) with scan-factor: 1000 searches up to 50,000 items and returns at most 50 results.

curl -X POST \
  -u username:password \
  -H "Content-Type: text/plain" \
  -H "scan-factor: 1000" \
  -d 'items.find({"name": {"$match": "*.deb"}}).include("name", "repo", "path").limit(10)' \
  "https://<JFrogPlatformURL>/artifactory/api/search/aql"

If you don't send a scan-factor greater than 1, Artifactory uses artifactory.aql.query.non.admin.scan.factor. The default is 1, so the search stays at .limit(N). A header value greater than 1 overrides the property for that request. The property applies to every user, including admins. On self-managed Artifactory, set it in artifactory.system.properties and restart Artifactory. On SaaS, send the scan-factor header with each request. For more information, see How Do I Get a Full .limit() Result?.

A higher value searches more items, so the query puts more load on Artifactory. Start with the smallest value that returns the results you need. If most of those extra items are in repositories you can't read, you can still get fewer than N results.

This applies when your query includes a .limit() greater than 0. If artifactory.aql.search.query.max.limit is set, keep .limit() lower than that value. It doesn't apply when Artifactory uses the embedded Derby database.

To get a full page, limit the query to repositories you can read:

items.find({
  "$or": [
    {"repo": "deb-readable-local"},
    {"repo": "deb-stable-local"}
  ],
  "name": {"$match": "*.deb"}
}).include("name", "repo", "path").limit(10)

You can also implement pagination when you want to focus on a subset of your results using theoffset element.

//Run the same example, but this time, display up to 50 items but skipping the first 100
items.find({"name" : {"$match":"*.jar"}).sort({"$asc" : ["repo","name"]}).offset(100).limit(50)

Unique Results (.distinct)

From Artifactory 7.37.x, you can disable unique results for improved performance when duplicates are acceptable:

items.find({"name": {"$match": "*.jar"}}).include("name").distinct(false)

Virtual Repository Field Behavior

The virtual_repos field lists the virtual repositories that contain the repository in which an item is stored.

• When your query targets a virtual repository (for example, items.find({"repo": "my-virtual"})), the virtual_repos field is automatically included in the output.
• For all other queries, you must explicitly request it using .include("virtual_repos").
• The virtual_repos field requires repo, path, and name to also be present in the result set (either as defaults or via .include()).

items.find({"repo": "libs-release-local"}).include("name", "repo", "path", "virtual_repos")

Range Notification Field

When the result set reaches the configured hard limit (aql.search.query.max.limit), the JSON response includes a notification field inside the range object:

{
    "range": {
        "start_pos": 0,
        "end_pos": 500000,
        "total": 500000,
        "notification": "AQL query reached the search hard limit, results are trimmed. Please address configurable limit."
    }
}

This notification indicates that additional matching results exist but were not returned. Narrow your query criteria or use pagination to retrieve additional results.

Output Limitations

The sort, limit, and offset methods only work when:

• Your query does not include an .include() method, OR
• Your .include() method only specifies fields from the primary domain

Example where sorting is ignored (sub-domain fields in .include()):

items.find()
  .include("artifact", "artifact.module", "artifact.module.build")
  .sort({"$asc": ["name"]})
  .limit(100)

Working alternative (restrict .include() to primary domain fields):

items.find()
  .include("name", "path", "repo")
  .sort({"$asc": ["name"]})
  .limit(100)

Artifactory checks what you can read after it searches, so .limit(N) can return fewer than N results. For more information, see Return the Results You Can Read.

Related Topics

• Query Execution and Permissions
• AQL Entities and Fields Reference
• AQL Examples and Common Patterns
• Search Criteria and Operators
• Query Structure and Syntax