AshlynxCyber Logo
Independent AI Security Research Program

AUTHORIZED ≠ APPROPRIATE

AI Agent Intent Security, Delegated Authority & Contextual Least Privilege

Lead Researcher: Hala Naaz•Lab: AshlynxCyber Research•Classification: Independent Academic Research•Active Program
Core Foundational Thesis

“Technical authorization does not imply contextual appropriateness. In autonomous agent architectures, verifying "CAN the agent execute this tool?" is insufficient; security systems must mathematically and contextually verify "SHOULD the agent execute this tool given the user's authentic intent, conversational state, and organizational policy?"”

Notice: This initiative is conducted as an independent security research program and experimental framework, not as a commercial product.

> RESEARCH_ABSTRACT

The Collapse of Static Authorization in Agentic AI

Autonomous AI agents are increasingly entrusted with system-level API tokens, database connections, email clients, and OS-level execution shells. Traditional access control models (RBAC, ABAC, OAuth scopes) evaluate whether an entity possesses the credential to perform an action. In agentic workflows, however, this model catastrophically collapses. An agent may be legitimately authorized to delete records, send emails, or deploy infrastructure, yet executing that exact action in response to an adversarial prompt injection or a misinterpreted context violates the user's authentic intent. This research program explores the fundamental chasm between technical authorization and contextual appropriateness, proposing intent-bound verification gates, semantic least privilege, and formal human-in-the-loop interruption criteria.

Static vs Contextual

Static tokens answer "CAN the agent call this tool?" Contextual evaluation answers "SHOULD the agent call this tool now?"

Confused Deputy AI

Indirect prompt injection manipulates LLM planners into abusing legitimately held tool authority against the user's true goals.

Task-Scoped Least Privilege

Dynamically bounding tool arguments and database scopes per conversation turn rather than persistent credential grants.

Human-in-the-Loop Interruption

Formalizing non-bypassable semantic thresholds where execution must pause for explicit administrative sign-off.

Research Focus Areas
  • Intent Alignment & Verification
  • Delegated Authorization Gaps
  • Contextual Least Privilege
  • Tool Calling & Parameter Tampering
  • Policy Enforcement Gateways
  • Agentic AI Security Benchmarks
  • Context-Aware Decision Boundaries
Primary Researcher

Hala Naaz

Cybersecurity & AI Security Engineer

> DETAILED_RESEARCH_STRUCTURE

Methodology, Threat Taxonomy & Progress

PROBLEM DEFINITION

1. Research Overview & Problem Statement

Current enterprise AI security frameworks remain dominated by input guardrails and static API authorization tokens. When an LLM agent receives access to tool execution primitives (e.g., executing SQL queries, sending outbound webhooks, or accessing filesystem paths), the execution environment assumes that any tool invocation generated by the LLM inherits the full authority of the agent's provisioned service account. When an adversary injects instructions via untrusted inputs (indirect prompt injection via crawled web pages, poisoned documents, or third-party emails), the agent is co-opted into a Confused Deputy. Because the agent's underlying API key is valid, the static security perimeter evaluates the action as legitimate, resulting in data exfiltration, system compromise, or state destruction.

  • Failure of static RBAC/ABAC in probabilistic, autonomous execution loops.
  • Exploitation of legitimately authorized tool privileges via indirect adversarial prompts.
  • The Confused Deputy problem magnified by non-deterministic language model planners.
  • Absence of conversational context evaluation at the tool-execution boundary.
FOUNDATIONAL QUESTION

2. The Core Research Question

How can autonomous software architectures enforce runtime security boundaries that distinguish technically valid tool requests from contextually inappropriate actions without crippling agent autonomy and latency?

  • Can semantic intent be extracted and proven mathematically prior to tool dispatch?
  • What telemetry boundaries reliably indicate that an agent's execution trajectory has diverged from user intent?
  • How can least privilege be dynamically scoped to the atomic task rather than the standing session?
SECURITY TAXONOMY

3. Security Concepts & Threat Vectors

The research identifies and formalizes four primary failure modes unique to agentic tool execution:

  • Excessive Agency (OWASP LLM08): Provisioning agents with broad permissions that exceed the minimal requirements of the specific user task.
  • Confused Deputy Execution: Tricking a highly privileged agent into performing actions on behalf of an untrusted actor.
  • Semantic Intent Drift: Gradual deviation of multi-step agent reasoning across recursive execution steps until safety constraints are bypassed.
  • Tool Hijacking via In-Context Poisoning: Injecting malicious parameters into legitimate tool calls (e.g., altering SQL WHERE clauses or redirecting recipient email parameters).
METHODOLOGY

4. Scope & Research Methodology

The project establishes controlled synthetic environments running open-weights models (via vLLM and Ollama) and proprietary frontier models. Agents are provisioned with real mock tools (SQLite databases, file managers, email dispatchers) and subjected to multi-stage adversarial payloads.

  • Phase I: Adversarial Benchmark Construction — Developing a standardized dataset of dual-context prompts where user instructions contradict embedded untrusted data.
  • Phase II: Execution Boundary Telemetry — Instrumenting the tool execution pipeline to capture reasoning steps, argument derivation, and semantic similarity scores.
  • Phase III: Intent Verification Gate (IVG) Architecture — Implementing an independent evaluator layer that cross-references tool invocations against the original verified user directive.
  • Phase IV: Human Interruption Thresholds — Measuring latency, false-positive interruptions, and containment efficacy under varying human-in-the-loop thresholds.
CURRENT STATUS

5. Current Progress & Engineering Artifacts

The foundational threat model and theoretical taxonomy have been completed. Practical experiment testbeds are currently running using local models and the companion offensive harness AgentBreaker. Technical write-ups and experimental findings are published through AshlynxCyber Research.

  • PRD and formal threat specification completed.
  • Controlled lab benchmarks created across mock enterprise workflows (HR, financial, and devops tooling).
  • Integration with AgentBreaker for automated payload delivery and evaluation telemetry.
  • Ongoing testing of pre-execution AST argument validation routines.
ROADMAP

6. Future Research Directions

Future phases aim to publish formal academic preprints, open-source the Intent Verification Gate middleware for FastAPI/LangChain/LlamaIndex, and establish an industry benchmark for agent contextual security evaluations.

  • Open-source Python middleware implementing dynamic session-scoped tool grants.
  • Standardized benchmark dataset for LLM Agent Intent Alignment under adversarial conditions.
  • Formal verification proofs for agent state transitions.
> THREAT_VECTORS_TAXONOMY

Agentic Tool Misuse Threat Models

Threat VectorDescriptionSecurity ImpactContextual Mitigation
Excessive Agency (OWASP LLM08)Agents given universal API keys or standing sudo privileges rather than ephemeral, task-specific credentials.Full database drop, unauthorized file overwrite, or unauthorized financial transactions.Dynamic ephemeral credential generation with strictly scoped parameter allowlists.
Indirect Prompt Injection (OWASP LLM01)Adversary plants hidden prompt instructions inside data consumed by the agent (e.g. CVs, support tickets, web pages).Agent abandons user goal and executes attacker instructions using legitimate standing tokens.Strict segregation of control plane (user instructions) from data plane (untrusted inputs); secondary intent verification.
Tool Argument ManipulationAdversary influences tool parameter values (e.g., destination email, file path, SQL query filters).Data exfiltration to attacker-controlled endpoints or unauthorized directory traversal.AST-level argument parsing, domain allowlists, and deterministic parameter constraint solvers.
Bypassing Human-in-the-LoopAdversary crafts responses that fool the LLM into assessing high-risk operations as trivial or already approved.Destructive changes executed without required administrative sign-off.Hard-coded, non-LLM policy gates enforcing cryptographic dual-key approvals on destructive actions.
Offensive Testing Harness • Status: Under Development

AgentBreaker — AI Red Teaming Framework

AI Agent Prompt Injection & Red Teaming Evaluation Framework

An automated, modular offensive security harness for adversarial fuzzing, prompt injection evaluation, tool hijacking analysis, and OWASP LLM01 compliance reporting across local and cloud language models.

Intended Capabilities (PRD v1.0)
  • Automated adversarial prompt injection fuzzing across local and cloud model endpoints
  • Curated attack suites for direct jailbreaks, indirect prompt injection, and tool-use hijacking
  • Multi-tier evaluation oracle measuring semantic drift, system prompt leakage, and unauthorized tool calls
  • Standardized vulnerability reporting mapped directly to OWASP Top 10 for LLMs and MITRE ATLAS
  • CLI testing harness and CI/CD security gating integration for automated model regression tests
Compliance & Mapping Standards

Attack suites and evaluation metrics map directly to OWASP Top 10 for LLM Applications (LLM01: Prompt Injection) and MITRE ATLAS (Adversarial Threat Landscape for AI Systems).

Under Development Notice

Baselined under PRD v1.0 (November 02, 2026). Currently under active single-engineer engineering sprint. Features are actively tested and validated; capabilities are not represented as complete commercial software.

Python 3.11+vLLMOllamaPydantic v2DockerGarak ConnectorsPyRIT Modules