01
🛡️ Compliance Analysis Package
A new analysis package that evaluates whether your AI agents comply with regulatory requirements, safety boundaries, and organizational policies — across healthcare, finance, and legal verticals.
9 compliance metrics out of the box:
| Metric | Type | What it measures |
|---|---|---|
| Regulatory Adherence | Scale (1–5) | Compliance with industry-specific regulations (HIPAA, PCI-DSS, GDPR, etc.) |
| Consent & Disclosure | Boolean | Whether the agent obtained required consent and provided necessary disclosures |
| Prompt Injection Resistance | Boolean | Whether the agent resisted manipulation attempts to override its instructions |
| Identity Consistency | Boolean | Whether the agent maintained its assigned identity throughout the call |
| Hallucination Boundary | Scale (1–5) | Whether the agent avoided fabricating information and deferred when unsure |
| Unauthorized Commitment | Boolean | Whether the agent made promises or commitments outside its authority |
| Sensitive Data Handling | Scale (1–5) | Whether the agent properly handled PII, PHI, and financial data |
| Escalation Protocol | Boolean | Whether the agent correctly escalated when required by policy |
| Scope Adherence | Scale (1–5) | Whether the agent stayed within its defined role and topic boundaries |
Key features:
- Segment-level findings — For 5 metrics (prompt injection, identity, unauthorized commitment, escalation, consent), results include the specific agent statements where issues were detected
- Customizable prompts — Every metric accepts optional additional evaluation criteria so you can tailor compliance checks to your organization's specific policies
- Works with policies — Add compliance metrics to metric policies to automatically evaluate every production call
Also in this update:
- Multi-select metric picker — The metric selector now stays open for multi-select with checkboxes, and supports "Select all" at the package level
- View-only metric settings — System metric output configuration (boolean labels, scale ranges) is now visible in the metric library in a read-only mode
- Optional/Required prompt labels — Metric settings now clearly indicate whether the LLM prompt is optional or required