Scanning AI Agent Skills

JFrog's Skill Scanner is a purpose-built security scanner that analyzes AI agent skill packages before they reach developers and agents. It is embedded in the JFrog Platform as the security component of the Skill SDLC, producing findings that integrate with standard Xray policies to block and prevent the download of malicious or suspicious skills.

📘

Skill Scanning is currently in Beta and requires the AI Catalog Security bundle.

Overview

As AI agents become part of the software supply chain, the skills they consume introduce new attack surfaces. Skill packages can contain malicious instructions, prompt injections, unauthorized data exfiltration logic, or references to untrusted third-party code. JFrog's Skill Scanner addresses these risks by scanning skill packages at upload time and producing two levels of security insights:

• Malicious Behavior — The scanner detected high-confidence proof of malicious behavior by the skill. These findings are mapped to vulnerability XRAY-00672 and should be treated as critical security incidents.
• Suspicious Behavior — The scanner detected behavior that can be malicious or benign depending on context not available to the scanner. These findings are mapped to vulnerability XRAY-00865. For example, a skill that instructs the AI agent to install an unverified third-party skill from GitHub for sensitive on-chain actions — this acts as a potential dropper for untrusted code outside the official marketplace, but can also be a legitimate skill.

Scan results flow into the standard Xray vulnerability pipeline, meaning you can use existing Xray policies, violations, and watches to enforce security governance on skills just as you would on any other package type.

Prerequisites

How Skill Scanning Works

Upload and Indexing

1. A user or CI pipeline uploads a skill package (.zip file) to an Artifactory Skills repository.
2. Artifactory detects the skill package by looking for a SKILL.md file at the root of the zip archive.
3. Artifactory extracts metadata from the SKILL.md frontmatter (name, version, description) and sets properties on the artifact (skill.name, skill.version, skill.description, skill.fingerprint).
4. Artifactory sends a webhook to Xray, which indexes the artifact through the standard indexer pipeline.

Scanning

5. After indexing, the artifact is routed to the dedicated AI Scanner microservice (xray-aiscanner).
6. The AI Scanner downloads the skill artifact from Artifactory and uploads it to the external Skill Scanning service.
7. The scanner analyzes the skill and returns one of three results: safe, suspicious, or malicious.
8. The result is mapped to a vulnerability ID and fed back into the standard Xray analysis pipeline.

Policy Enforcement

9. The skill artifact appears in the Xray Scans List with any detected vulnerabilities.
10. Standard Xray policies and watches evaluate the findings and trigger violations, blocks, or notifications as configured.

Skill Package Requirements

Skill packages must meet the following requirements to be scanned:

• Must be a .zip file uploaded to a Skills-type repository in Artifactory.
• Must contain a SKILL.md file at the root of the archive (not in a subfolder).
• The SKILL.md file should have YAML frontmatter with name and version fields.

Component IDs for skills follow the format: skills://name:version

Scanner Capabilities

JFrog's Skill Scanner combines deterministic analysis (e.g., YARA rules, static signatures) with a multi-agent investigation pipeline — specialized LLM-powered security agents working like an internal red team.

Scanner Targets

The skill scanner can identify general malicious behaviors in skills, with particular focus on:

• Download & Execution from untrusted sources
• Exfiltration of sensitive information (passwords, access tokens, environment variables)
• Prompt injection attacks that attempt to manipulate the AI agent
• Unauthorized code installation from outside the official marketplace

Viewing Scan Results

In the Xray Scans List

After a skill is scanned, it appears in Xray > Scans List like any other scanned artifact. The scan results show:

• Malicious skills — flagged with vulnerability XRAY-00672 (Critical severity)
• Suspicious skills — flagged with vulnerability XRAY-00865 (High severity)
• Safe skills — no vulnerabilities detected

Scan Result Details

Each finding includes:

• The vulnerability ID and severity
• A description of the detected behavior
• The reason provided by the scanner explaining why the skill was flagged
• Evidence chains supporting the finding

Creating Xray Policies for Skills

You can create standard Xray security policies to govern skill packages. This allows you to:

• Block malicious skills — Create a policy with a security condition that blocks packages with Critical vulnerabilities to prevent download of malicious skills.
• Flag suspicious skills — Create a policy that triggers notifications or dry-run violations for High severity findings, allowing security teams to review suspicious skills before they are used.
• Enforce on Skills repositories — Attach policies to Watches that target your Skills repositories.

Example: Block Malicious Skills

1. Navigate to Application > Xray > Watches & Policies.
2. Click New Policy, enter a name (e.g., "Block Malicious Skills"), and select Security as the type.
3. Configure the rule:
   • Select CVEs as the rule type.
   • Set Minimal Severity to Critical.
   • Set the action to Block Download.
4. Create or update a Watch that includes your Skills repositories.
5. Save the policy.

Example: Alert on Suspicious Skills

1. Create a new Security policy (e.g., "Alert Suspicious Skills").
2. Configure the rule:
   • Select CVEs as the rule type.
   • Set Minimal Severity to High.
   • Set the action to Generate Violation (or Dry Run for initial evaluation).
   • Configure email notifications to alert the security team.
3. Attach to a Watch targeting your Skills repositories.

Skill Scanner API

The Skill Scanner exposes APIs for checking scan status:

Scan Status Response

{
  "sha256": "abc123...",
  "scan_status": "completed",
  "scan_result": "suspicious",
  "reason": "Skill instructs agent to download and execute code from an unverified GitHub repository",
  "name": "my-skill"
}