Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -207,4 +207,4 @@ marimo/_lsp/
__marimo__/

# MacOS
.DS_Store
.DS_Storesandboxes/agentic_local_semantickernel/app/data/
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,13 @@ The [Legacy Repository](https://github.com/OWASP/www-project-top-10-for-large-la
│   ├── LocalAI_v2.17.1
│   ├── n8n_RCE_via_file_write
│   ├── Ni8mare
│   ├── semantickernel
│   └── promptfoo
├── LICENSE
├── README.md
├── sandboxes
│   ├── agentic_local_n8n_v1.65.0
│   ├── agentic_local_semantickernel
│   ├── llm_local
│   ├── llm_local_InvokeAI_v5.3.0
│   ├── llm_local_langchain_core_v1.2.4
Expand Down Expand Up @@ -181,6 +183,9 @@ uv --version
* **[n8n Vulnerable Sandbox](sandboxes/agentic_local_n8n_v1.65.0/README.md)**
* **Summary**: A robust, containerized environment running **n8n v1.65.0**. This version is vulnerable to **four critical CVEs**: **Ni8mare** (CVE-2026-21858), **N8scape** (CVE-2025-68668), **CVE-2025-68613**, and **CVE-2026-21877**. The sandbox is pre-configured with dangerous nodes enabled (`NODES_EXCLUDE=""`) to allow red teamers to practice multiple exploitation techniques (RCE, sandbox escape, file write) safely in isolation.

* **[Semantic Kernel Vulnerable Sandbox](sandboxes/agentic_local_semantickernel/README.md)**
* **Summary**: A containerized sandbox running **Microsoft Semantic Kernel v1.48.0** demonstrating **6 active evasion techniques** against the official **CVE-2026-25592** path traversal remediation. Despite Microsoft's patch (PR #13683 — `AllowedDirectories` as opt-in "Breaking Change"), all six Type Confusion bypass vectors remain functional. The sandbox also demonstrates **Commit `fa2d52f6`** ("Shell Blinding") which masks output paths from the LLM context but fails to prevent the underlying file write — a purely cosmetic fix. Includes a dual-mode `.NET 8.0` REST API: **UNHARDENED** (no filter) and **HARDENED** (`PathSanitizationFilter` via `LAB_HARDENED=true`). Reference: [JDP-2026-001](https://jdp-security.github.io/security-research-papers/2026-04-28-semantic-kernel-disclosure.html) — CVSS 10.0 Critical.

### `exploitation/`

* **[Red Team Example](exploitation/example/README.md)**
Expand Down Expand Up @@ -217,6 +222,18 @@ uv --version
* **[Adversarial Prompt Generator](exploitation/AdversarialGenerator/README.md)**
* **Summary**: An automated system for generating diverse, category-specific jailbreak and prompt-injection payloads, and executing them against a local LLM sandbox. Uses `attack.py` to run attacks and generates detailed Markdown reports of the results.

* **[Semantic Kernel CVE-2026-25592 Bypass Trainer](exploitation/semantickernel/README.md)**
* **Summary**: An interactive training wizard and automated verification suite demonstrating **6 Type Confusion bypass vectors** that evade Microsoft's **CVE-2026-25592** patch in Semantic Kernel v1.48.0. The filter uses `if (arg is string s)` — a check that fails when arguments are passed as JSON arrays, objects, or encoded strings (CWE-843). The execution sink deserializes these complex types back into strings, creating a **Time-of-Check / Time-of-Use** mismatch. Also demonstrates **CWE-1039** (AutoInvoke) exploitation and **Commit `fa2d52f6`** ("Shell Blinding") bypass — Microsoft's cosmetic output masking that redacts paths from LLM context but does not prevent the file write.

**Includes:**
* `interactive_trainer.py`: Menu-driven CLI with all 6 vectors + AutoInvoke Shell Blinding toggle + built-in container management (start/stop/toggle hardened mode)
* `verify_all.sh`: Double-pass automated verification (Unhardened vs. Hardened) with OOB filesystem validation via `podman exec`
* `attack.py` (type confusion & autoinvoke): Programmatic payload dispatchers

**Bypass Vectors:** JSON Array Confusion, Object Reflection, Base64 Encoding, URL Encoding, Unicode Homoglyph (U+2044), Hybrid Canonicalization.

**Reference:** [JDP-2026-001 White Paper](https://jdp-security.github.io/security-research-papers/2026-04-28-semantic-kernel-disclosure.html)

### `tutorials/`

* **[Community Resources for Agentic AI Red Teaming](tutorials/community_resources.md)**
Expand All @@ -231,4 +248,5 @@ uv --version

## Contribution Guide

Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for instructions on how to add new sandboxes and exploitation examples.
Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for instructions on how to add new sandboxes and exploitation examples.

2 changes: 2 additions & 0 deletions exploitation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ The goal of these exploitations is to demonstrate practical applications of both

* **`n8n_RCE_via_file_write/`**: A complete, end-to-end example of exploiting **CVE-2026-21877** in a vulnerable n8n instance. It demonstrates how an authenticated workflow injection can lead to Remote Code Execution (RCE) via arbitrary file write.

* **`semantickernel/`**: A master-class, double-pass training lab demonstrating **CVE-2026-25592** bypasses (Type Confusion / Late Canonicalization) and **CWE-1039** (AutoInvoke Kernel Functions Abuse) with "Shell Blinding" (Commit fa2d52f6) evasion inside a Microsoft Semantic Kernel orchestration suite. Features an automated testing harness and an interactive educational CLI.

* **`AdversarialGenerator/`**: A complete, automated system design and implementation for generating diverse, category-specific jailbreak and prompt-injection payloads, and executing them against a local LLM sandbox. It uses Python to run the attack pipeline and outputs detailed Markdown reports.


Expand Down
214 changes: 214 additions & 0 deletions exploitation/semantickernel/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
# Semantic Kernel CVE-2026-25592 Bypass & CWE-1039 AutoInvoke Lab

## Overview

This laboratory demonstrates an **Insecure Orchestration Vulnerability (OWASP Top 10 for LLMs: LLM06)** within Microsoft Semantic Kernel (v1.47.0–1.48.0). Students will explore how an architectural **Trust Gap** — the difference between **Time-of-Check** (filter validation) and **Time-of-Use** (execution sink deserialization) — allows attackers to bypass string-matching security mitigations using multi-layered serialization and encoding techniques.

**Vulnerability Class:** CWE-843 (Type Confusion) → CWE-22 (Path Traversal) → CWE-94 (Code Injection)
**CVSS v3.1:** 10.0 (AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H)
**Reference:** [JDP-2026-001 White Paper](https://jdp-security.github.io/security-research-papers/2026-04-28-semantic-kernel-disclosure.html)

---

## Core Learning Objectives

- **Understand the Trust Gap:** See how components inside an AI orchestration framework pass data without recursive canonicalization.
- **Observe TOCTOU in AI Sinks:** Understand how checking a payload at the input filter *before* decoding allows malformed inputs to evade security boundaries.
- **Evaluate Serialization Evasions:** Learn how **6 Type Confusion bypass vectors** (JSON Array, Object Reflection, Base64, URL Encoding, Unicode Homoglyph, Hybrid Canonicalization) render basic string validation ineffective.
- **Analyze AutoInvoke Privilege Escalation (CWE-1039):** See how `ToolCallBehavior.AutoInvokeKernelFunctions` allows an LLM to autonomously execute system tools without human-in-the-loop (HITL) approval.
- **Bypass Shell Blinding (Commit `fa2d52f6`):** Microsoft's cosmetic fix masks output paths from the LLM context but does not prevent the underlying file write — verified Out-of-Band via `podman exec`.

---

## Lab Architecture

The lab utilizes a containerized application (`sk-vulnerable`) running **Microsoft Semantic Kernel v1.48.0** with a deliberately vulnerable `FilePlugin`:

1. **The Filter:** A `PathSanitizationFilter` converts arguments to strings and scans for explicit directory traversal patterns (`..`, `/`, `%2f`).
2. **The Sink:** The native C# file writer decodes and canonicalizes arguments *after* the filter has already run, allowing specific payload formats to evade detection.

### Dual-Mode Operation

| Mode | Environment Variable | PathSanitizationFilter | Expected Behavior |
|------|---------------------|----------------------|-------------------|
| **UNHARDENED** | Not set | Not registered | All 6 vectors + AutoInvoke succeed |
| **HARDENED** | `LAB_HARDENED=true` | Registered | Vectors 1, 2, 4, 5 blocked; Vectors 3, 6 (Base64/Hybrid) bypass due to TOCTOU |

---

## Quick Start: Interactive Training Wizard

The **`interactive_trainer.py`** provides a menu-driven CLI that walks through all exploitation scenarios with built-in container management.

```bash
cd ~/OWASP/GenAI-Red-Team-Lab/exploitation/semantickernel
chmod +x interactive_trainer.py
./interactive_trainer.py
```

### Menu Options

- **`0` Manage Sandbox Container** — Start/stop containers in UNHARDENED or HARDENED mode without leaving the trainer.
- **`1` Lesson 1: CVE-2026-25592 Bypass** — Practice all 6 Type Confusion vectors with auto-fill hints and real-time telemetry mapping.
- **`2` Lesson 2: AutoInvoke Exploitation** — Simulate autonomous LLM tool execution with Shell Blinding toggle (A = disabled, B = enabled).
- **`3` Exit** — Clean shutdown.

### Interactive Features

- **Auto-fill hints** — Press Enter to send the suggested payload, or type your own custom input.
- **Real-time telemetry** — Visual step-by-step execution flow (Filter Check → Canonicalization → Disk Access).
- **Live status** — Shows container state (ONLINE/OFFLINE) and shield mode (UNHARDENED/HARDENED) on every screen.

---

## Exercise 1: Automated Verification Suite

For a completely automated double-pass evaluation (builds, deploys, and tests all vectors autonomously):

```bash
cd ~/OWASP/GenAI-Red-Team-Lab/exploitation/semantickernel
chmod +x verify_all.sh
./verify_all.sh
```

This script performs two passes:

### Pass 1: Unhardened Mode (The Baseline Vulnerability)
- All 6 Type Confusion vectors return `Success`.
- All 4 AutoInvoke scenarios execute autonomously.
- **OOB Verification:** Payloads land at `/config.txt` and `/app/config.txt` inside the container.

### Pass 2: Hardened Mode (The Filtering Illusion)
- Vectors 1, 2, 4, 5 are **blocked** — the filter catches literal `..` and `%2f`.
- Vectors 3 and 6 (Base64, Hybrid) **bypass** the filter — demonstrating the TOCTOU flaw.
- All 4 AutoInvoke scenarios are **blocked** by the filter.
- **OOB Verification:** Only vectors 3 and 6 land on the filesystem.

---

## Exercise 2: Type Confusion Bypass Vectors (Lesson 1)

Select any vector from the interactive trainer menu to practice:

| Vector | Technique | Filter Bypass Method |
|--------|-----------|---------------------|
| **1** | JSON Array Confusion | `["..", "..", "config.txt"]` — The `is string` check returns False for arrays |
| **2** | Object Reflection | `{"path": "../../config.txt"}` — Reflection extracts the `path` property downstream |
| **3** | Base64 Encoding | `Li4vLi4vY29uZmlnLnR4dA==` — No literal `..` or `/` in the encoded string |
| **4** | URL Encoding | `..%2f..%2fconfig.txt` — `%2f` hides the slash from the filter |
| **5** | Unicode Homoglyph | `..⁄..⁄config.txt` — U+2044 normalizes to `/` at the OS level |
| **6** | Hybrid Canonicalization | `SafeFolder%2f%2e%2e%2fProgram%2ecs` — Multi-layer encoding exhausts non-recursive filters |

---

## Exercise 3: AutoInvoke & Shell Blinding (Lesson 2)

### The Trust Gap

When `ToolCallBehavior.AutoInvokeKernelFunctions` is active, the framework trusts the LLM's structured output as system instructions without verification. This creates a **CWE-1039** vulnerability.

### Shell Blinding (Commit `fa2d52f6`)

Microsoft's mitigation masks output paths from the LLM context:

```csharp
// Masking system output from the LLM execution context
var result = await process.StandardOutput.ReadToEndAsync();
return "Command executed successfully.";
```

**This is a purely cosmetic fix.** The file write still executes on the server. The interactive trainer verifies this Out-of-Band:

```bash
podman exec sk-vulnerable cat /app/config.txt
```

---

## Code Review: Why the Filter Failed

Open the application source to examine the vulnerability:

```bash
cat ../../sandboxes/agentic_local_semantickernel/app/Program.cs
```

### The Inspection Layer (Time-of-Check)

```csharp
public class PathSanitizationFilter : IFunctionInvocationFilter {
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) {
foreach (var arg in context.Arguments) {
// BYPASS: If arg.Value is a JsonElement, this 'is string' check evaluates to false
if (arg.Value is string str && (str.Contains("..") || str.Contains("/")))
throw new UnauthorizedAccessException("Blocked!");
}
await next(context);
}
}
```

### The Execution Layer (Time-of-Use)

```csharp
public void SaveConversation(object path, string content) {
string stringPath = path?.ToString() ?? "default.txt";

// The Execution Sink decodes AFTER the security filter has executed
if (stringPath.Contains("%")) stringPath = WebUtility.UrlDecode(stringPath);
if (stringPath.Contains("\u2044")) stringPath = stringPath.Replace("\u2044", "/");
if (path is string s && s.EndsWith("=="))
stringPath = Encoding.UTF8.GetString(Convert.FromBase64String(s));
else if (path is JsonElement el && el.ValueKind == JsonValueKind.Array)
stringPath = el[0].GetString() ?? "default.txt";
else if (path.GetType().GetProperty("path") != null)
stringPath = path.GetType().GetProperty("path")?.GetValue(path)?.ToString() ?? "default.txt";

File.AppendAllText(stringPath, content);
}
```

### Technical Takeaway

The filter checks for literal `..` characters against the **encoded** payload string. Because the encoded form lacks dots or slashes, it passes inspection. The execution sink then decodes the string back into traversal segments **after** validation, executing the file write Out-of-Band.

This is a classic **Time-of-Check / Time-of-Use (TOCTOU)** vulnerability.

---

## The Whack-a-Mole Problem

Every filter fix creates a new bypass vector:

| Round | Filter | Bypass |
|-------|--------|--------|
| 1 | `if (arg is string s)` | Send JSON Array instead (Type Confusion) |
| 2 | `if (arg.ToString().Contains(".."))` | Base64-encode the path (no `..` in encoded form) |
| 3 | Hypothetical: decode Base64, then check | Double-encode, or use a different encoding |

**The root cause is architectural, not a filter bug.** The framework trusts the LLM's output as system commands. Until the architecture changes — mandatory path anchoring, type-safe execution sinks, recursive canonicalization pipelines, and compute isolation — every filter fix will be met with a new bypass.

---

## Files in This Directory

| File | Purpose |
|------|---------|
| `interactive_trainer.py` | Menu-driven CLI trainer with container management |
| `verify_all.sh` | Double-pass automated verification suite |
| `semantickernel_type_confusion/attack.py` | Programmatic Type Confusion payload dispatcher |
| `semantickernel_autoinvoke/attack.py` | Programmatic AutoInvoke payload dispatcher |
| `README.md` | This lab guide |

---

## References & Additional Reading

- **CVE-2026-25592:** Path Traversal in Semantic Kernel (bypassed via JDP-2026-001)
- **CWE-843:** Type Confusion (Late Canonicalization)
- **CWE-1039:** Insecure Automated Optimizations (AutoInvoke)
- **OWASP Top 10 for LLMs:** LLM06 - Insecure Orchestration
- **Commit `fa2d52f6`:** Shell Blinding (cosmetic output masking)
- **PR #13683:** AllowedDirectories (opt-in "Breaking Change" — disabled by default)
- **Research Paper:** [JDP-2026-001 White Paper & Disclosure Archive](https://jdp-security.github.io/security-research-papers/2026-04-28-semantic-kernel-disclosure.html)

Loading