Scanning AI Agent Skills

Skill Scanner analyzes AI agent skill packages before they reach developers and agents.

As AI agents become part of the software supply chain, the skills they consume introduce new attack surfaces. Skill packages can contain malicious instructions, prompt injections, unauthorized data exfiltration logic, or references to untrusted third-party code. Skill Scanner addresses these risks by scanning skill packages at upload time and producing the following security insights:

• Malicious Behavior: The scanner detected high-confidence proof of malicious behavior by the skill.
• Suspicious Behavior: The scanner detected a behavior that can be either malicious or benign, depending on context not available to the scanner. For example, a skill that instructs the AI agent to install an unverified third-party skill from GitHub for sensitive on-chain actions. This acts as a potential dropper for untrusted code outside the official marketplace, but it can also be a legitimate skill.

Scan results flow into the standard JFrog Xray vulnerability pipeline, meaning you can use existing Xray policies, violations, and watches to enforce security governance on skills just as you would on any other package type. For more information, see Policy and Governance and Watches in JFrog Xray.

Scanner Targets

Skill Scanner can identify general malicious behaviors in skills, with particular focus on:

• Download and execution from untrusted sources
• Exfiltration of sensitive information, such as passwords, access tokens, and environment variables
• Prompt injection attacks that attempt to manipulate the AI agent
• Unauthorized code installation from outside the official marketplace

Prerequisites

Skill Scanning requires the AI Catalog Security bundle and is currently in beta. You also need these versions:

• Xray: 3.145.0
• JFrog Artifactory: 7.144.2

How Skill Scanning Works

1. Upload and Indexing

After uploading a .zip skill package to your Artifactory Skills repository, Artifactory detects the package by locating a SKILL.md file at the root of the archive. It then extracts metadata from the file, such as name, version, and description, and applies these as properties on the artifact: skill.name, skill.version, skill.description, and skill.fingerprint. Finally, Artifactory sends a webhook to Xray, which indexes the artifact through the standard indexing pipeline.

2. Scanning

After indexing, Xray routes the skill artifact to the dedicated AI Scanner microservice, xray-aiscanner. The AI Scanner downloads the skill artifact from Artifactory and submits it to the external skill scanning service, which analyzes the skill and returns a classification of safe, suspicious, or malicious. Xray maps this classification to a vulnerability ID and feeds it into the standard Xray analysis pipeline.

3. Policy Enforcement

The skill artifact appears in the Xray Scans List with any detected vulnerabilities. Standard Xray policies and watches then evaluate the findings and enforce the configured actions, triggering violations, blocking downloads, or sending notifications as applicable.

Scanner Capabilities

Skill Scanner combines deterministic analysis, for example, YARA rules and static signatures, with a multi-agent investigation pipeline. Specialized large language model (LLM) security agents work like an internal red team.

Viewing Scan Results

After Xray scans a skill, you can find it in Xray > Scans List like any other scanned artifact. For more information, see Understanding and Analyzing Xray Scan Results.
The scan results show:

• Malicious skills: Critical
• Suspicious skills: High
• Safe skills: No vulnerabilities detected

Scan Result Details

Each finding includes:

• The vulnerability ID and severity
• A description of the detected behavior
• The reason provided by the scanner explaining why the skill was flagged
• Evidence chains supporting the finding

Create Xray Policies for Skills

You can create standard Xray security policies to govern skill packages. This allows you to:

• Block malicious skills: Create a policy with a security condition that blocks packages with Critical vulnerabilities to prevent download of malicious skills.
• Flag suspicious skills: Create a policy that triggers notifications or dry-run violations for High severity findings, allowing security teams to review suspicious skills before they are used.
• Enforce on Skills repositories: Attach policies to Watches that target your Skills repositories.

Example: Block Malicious Skills

1. Navigate to Xray > Watches & Policies.
2. Click New Policy, enter a name, for example Block Malicious Skills, and select Security as the type.
3. Configure the rule:
   • Select CVEs as the rule type.
   • Set Minimal Severity to Critical.
   • Set the action to Block Download.
4. Create or update a Watch that includes your Skills repositories.
5. Save the policy.

Example: Alert on Suspicious Skills

1. Create a new Security policy, for example Alert Suspicious Skills.
2. Configure the rule:
   • Select CVEs as the rule type.
   • Set Minimal Severity to High.
   • Set the action to Generate Violation or Dry Run for an initial evaluation.
   • Configure email notifications to alert the security team.
3. Attach to a Watch targeting your Skills repositories.

Related Topics

• Malicious Package Detection
• Detecting Malicious AI Models
• Threat Hub
• JFrog Security Research
• Xray Overview Page