Guardrail sandwich pattern: checks before and after the model
Put input guardrails before the model (mask personal data, detect prompt injection and jailbreaks, enforce scope) and output guardrails after it (validate the schema, check for leaked data and harmful content, check grounding for high-risk answers), then restore any masked data the user should see. The two slices do different jobs: input checks protect the model and your data, output checks protect the user and your company. Keep the inline checks fast, and send slower checks to background evaluators.
Why both slices
Without input checks, personal data and injected instructions reach the model. Without output checks, broken JSON, leaked system prompts and harmful text reach the user. One slice can't do the other's job: an input filter can't tell whether the answer will leak the system prompt, and an output filter can't take back data you already sent to a provider.
Layers
| Slice | Checks | Typical tools |
|---|---|---|
| Input | PII masking, injection and jailbreak detection, scope and policy | Microsoft Presidio, small classifiers, NVIDIA NeMo Guardrails |
| Model | Your model call | |
| Output | Schema validation, leakage, toxicity, grounding | Pydantic, Llama Guard or similar classifiers, validated judges |
| Restore | Unmask personal data the user should see | The reverse map from the input step |
Small language models, or masked language models like BERT trained for one check, make fast and cheap guardrails. They can classify an input as an injection attempt in milliseconds, where a large model would take a second.
Masking with a reverse map
Mask personal data before the model call, keep a map from placeholders to the original values, and restore them in the final answer. The model works with placeholders, the user sees their real details, and the provider never receives them.
import json
import re
PATTERNS = {
"EMAIL": re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+"),
"PHONE": re.compile(r"\+?\d[\d -]{8,}\d"),
}
def mask(text: str) -> tuple[str, dict[str, str]]:
mapping = {}
for label, pattern in PATTERNS.items():
for i, value in enumerate(dict.fromkeys(pattern.findall(text)), start=1):
token = f"<{label}_{i}>"
mapping[token] = value
text = text.replace(value, token)
return text, mapping
def unmask(text: str, mapping: dict[str, str]) -> str:
for token, value in mapping.items():
text = text.replace(token, value)
return text
def validate(raw: str) -> dict:
data = json.loads(raw)
if set(data) != {"reply", "needs_human"} or not isinstance(data["needs_human"], bool):
raise ValueError("schema mismatch")
return data
user = "My email is priya@example.com and phone +91 98765 43210, please update my address."
masked, mapping = mask(user)
print("to model:", masked)
model_output = '{"reply": "I have sent a confirmation link to <EMAIL_1>.", "needs_human": false}'
answer = validate(model_output)
print("to user: ", unmask(answer["reply"], mapping))
In production, use a dedicated PII library like Presidio, which detects many more entity types with context, and keep the mapping only for the life of the request.
Inline or later
Inline checks add latency to every request, so keep them fast: rules, regexes, schema checks and small classifiers. Slower checks, like an LLM judge for policy correctness, run on a sample in the background, or inline only for high-risk routes. See guardrails vs evaluators.
What to implement first
- PII masking with a reverse map.
- Schema validation of structured outputs.
They catch the most common real failures. Then add injection detection for any input that comes from users or external content, and output filters for leakage and harmful content.
Measuring it
Track the trigger rate and the false positive rate of each guardrail, and review them weekly. A guardrail that fires too often blocks real users, and one that never fires may not be working.
From the floor
While researching a legal-tech problem, my first idea was filters as a service: a library of input and output filters, like safety filters used during LLM inference, built for industries where compliance is the top priority, such as law, finance and healthcare. That research led to the hallucination citation verifier, which is a specialised output filter: it checks whether each cited case actually says what the answer claims.
In the production pipeline I teach, the sandwich is zones 2 and 6: Presidio masks personal data and NeMo Guardrails checks for injection before the model, and Pydantic validates the output and Llama Guard filters leakage and toxicity after it, with the masked data mapped back before the answer leaves.