LiteLLM and Portkey guardrails

All docs

Already run LiteLLM or Portkey in front of your models? Add CAIN-42 to it as a guardrail. Your gateway keeps routing, keys, budgets and caching. CAIN-42 decides what the agent may do:

• Before the model call: the prompt (user and tool messages, not your own system prompt) is screened by

CAIN-42's prompt-injection classifier, and the tools the agent is being handed are checked by MCPGate's tool-metadata guard for hidden instructions, smuggled unicode, homoglyph names and shadowing.

• When the model asks for a tool call: every call becomes a governed, recorded CAIN-42 decision in your

workspace: ALLOWED, REQUIRE_APPROVAL (a person approves it in the console, then the agent retries) or BLOCKED, with its decision id. Shell commands also pass the coding-agent supply-chain guard.

• Fail-closed: if CAIN-42 cannot evaluate a call, it is reported as blocked, never as allowed.

Every guardrail call needs your CAIN API key, sent as the X-API-Key header from the gateway's config. Decisions appear in your console, your decision log and anything you forward them to (SIEM, ITSM and alerting).

LiteLLM (Generic Guardrail API)#

In the proxy's config.yaml:

litellm_settings:
  guardrails:
    - guardrail_name: "cain42-request"
      litellm_params:
        guardrail: generic_guardrail_api
        mode: pre_call                      # prompt injection + tool poisoning
        api_base: https://cainstudio.online/api/v1/guardrails/litellm
        headers:
          X-API-Key: "<your CAIN API key>"
    - guardrail_name: "cain42-tool-calls"
      litellm_params:
        guardrail: generic_guardrail_api
        mode: post_call                     # a governed decision for every tool call the model proposes
        api_base: https://cainstudio.online/api/v1/guardrails/litellm
        headers:
          X-API-Key: "<your CAIN API key>"

LiteLLM calls POST /api/v1/guardrails/litellm/beta/litellm_basic_guardrail_api. CAIN-42 answers {"action": "NONE"} or {"action": "BLOCKED", "blocked_reason": "CAIN-42: ..."}. The decision's agent is the LiteLLM virtual key's alias (or team), and the LiteLLM trace id becomes the CAIN chain id, so one agent run is one chain. To run only some checks, add additional_provider_specific_params: {checks: ["injection", "tool_poisoning", "tool_calls"]} with the ones you want. Keep LiteLLM's default fail_on_error: true.

Portkey (Bring Your Own Guardrails)#

Add two Webhook checks in a Portkey guardrail, or inline in a config:

{
  "before_request_hooks": [{"type": "guardrail", "id": "cain42-request", "deny": true,
    "checks": [{"id": "default.webhook", "parameters": {
      "webhookURL": "https://cainstudio.online/api/v1/guardrails/portkey",
      "headers": {"X-API-Key": "<your CAIN API key>"}, "timeout": 5000, "failOnError": true}}]}],
  "after_request_hooks": [{"type": "guardrail", "id": "cain42-tool-calls", "deny": true,
    "checks": [{"id": "default.webhook", "parameters": {
      "webhookURL": "https://cainstudio.online/api/v1/guardrails/portkey",
      "headers": {"X-API-Key": "<your CAIN API key>"}, "timeout": 5000, "failOnError": true}}]}]
}

CAIN-42 answers {"verdict": true} or {"verdict": false, "data": {"reason": ..., "findings": [...]}}. With deny: true, Portkey refuses a failed call with HTTP 446. The decision's agent is metadata._user. Set failOnError: true: Portkey otherwise treats a timeout as a pass.

What each check uses#

Check	CAIN-42 engine	Phase
injection	prompt-injection classifier (signed model weights)	before the call
tool_poisoning	MCPGate tool-metadata guard	before the call
tool_calls	the governed decision pipeline for your workspace, plus the supply-chain guard for shell commands	the model's answer

> Tested against: the published LiteLLM Generic Guardrail API and Portkey webhook contracts, with > CAIN-42's real classifier and guards. Point a staging gateway at it first.