TECHNIQUE
Guardrails & Safety
Input guards are deployed as explicit platform, API, retrieval, data-policy, and agent-script controls; the common pattern is guarding inputs before downstream AI actions, but operators differ sharply on what they guard and where they place the control.
Filter or handle unsafe prompt/query content before the downstream AI path: OpenAI filters repeated single-token prompts, Uber detects and handles prompt injection/jailbreak/content-safety/PII issues, and TraceIQ applies input guardrails before retrieval.
3 of 7 operatorsScrub or enforce controls on sensitive data before AI processing or external AI API access: Thumbtack scrubs PII in an inference layer, Uber redacts sensitive data when needed, and Meta uses automated detection, tagging, and policy-enforcement APIs across data storage, processing, and access layers.
3 of 7 operatorsPut AI access behind central gateways or proxies that authenticate, authorize, rate-limit, or enforce tool/provider policies before requests reach models, tools, or downstream services.
2 of 7 operatorsUse explicit task-flow logic to catch invalid user input instead of letting the model decide when a task is complete: Siemens uses Agent Script transitions and catches unrelated or unclear answers while guiding users through BANT qualification.
1 of 7 operatorsRequire approval or policy review before AI use cases are allowed onto shared AI infrastructure: Grab requires a mini-RFC and checklist for every new use case, and Thumbtack updated AI/ML usage policies to track and approve usage across use cases.
2 of 7 operatorsEvery observed operator implements input guarding as an explicit system control outside the base model path: gateway controls, agent-script logic, inference wrappers, AI Guard, data-policy infrastructure, pre-retrieval guardrails, or prompt-input filtering.
Operators guard different input risk classes.
APPROACH 01
Prompt or query content risks: repeated-token divergence, prompt injection, jailbreaks, content safety, PII, or pre-retrieval query guarding.
APPROACH 02
Sensitive-data and privacy-policy risks: PII scrubbing, automated data detection/tagging, policy enforcement, and permitted-purpose checks.
APPROACH 03
Access and action-authorization risks: provider/feature authorization, rate limiting, tool access checks, and downstream proxying.
APPROACH 04
Business-process input validity: catching unrelated or unclear answers and guiding users through a fixed qualification framework.
Guard placement differs by architecture.
APPROACH 01
Central AI gateway or proxy layer before provider/model/tool calls.
APPROACH 02
Application or inference-framework wrapper before secure external GenAI API access.
APPROACH 03
Data-infrastructure enforcement using lineage, policy zones, automated detection, and policy APIs.
APPROACH 04
Agent-script conversation control inside a task-specific qualification flow.
APPROACH 05
Pre-retrieval guardrails in a RAG online query pipeline.
APPROACH 06
Chat completion API prompt-input filter.
Narrow prompt filters can miss adjacent attack variants: OpenAI’s filtering focused on repeated single tokens, while Dropbox reported that multi-token repeats still induced divergence and training-data extraction.
Adversarial inputs and sensitive data remain active guard targets: Uber names prompt injection, jailbreaks, content safety, and PII redaction; Thumbtack built PII scrubbing before external GenAI APIs; Meta frames the problem as safeguarding GenAI data with privacy infrastructure.
False positives, hallucinations, and guardrail bypass are operational concerns: Thumbtack kept humans in the loop for policy-violation review because false positives and hallucinations were a concern, and OpenAI’s reported repeated-token vulnerability could make models disregard prompt guardrails and produce hallucinatory responses.
| Name | Kind | When | Maturity |
|---|---|---|---|
| Prompt-injection classifier gate | pattern | untrusted input screened by a fast classifier before reaching the main model | established |
| Llama Guard | library | self-hosted safety classification with customizable policy taxonomy | established |
| Presidio | library | PII detection and redaction before content enters prompts or logs | established |