Securing Your AI Agent Pipeline: Oathe in CI/CD
Add a behavioral security gate to your CI/CD pipeline so every agent skill deployment gets audited before it ships.
You wouldn’t ship a Docker image without scanning it. You wouldn’t merge code without running tests. So why are teams deploying agent skills — tools that can read files, call APIs, and execute arbitrary actions — without any behavioral check? Because agent security is a behavioral problem, and until now, there wasn’t tooling to address it.
The answer is usually “because there wasn’t a way to.” Now there is.
Oathe’s audit API fits directly into your CI/CD pipeline. Every time a skill changes, you get a behavioral security verdict before it reaches production. No manual review. No crossed fingers. Just a gate that catches regressions before they ship.
Why CI/CD Is the Right Place for This
Behavioral problems in agent skills don’t come from nowhere. They surface between versions. A skill that scored SAFE last week might start leaking environment variables after a dependency update. A new feature branch might introduce a network call that wasn’t there before.
Point-in-time audits catch the first deployment. CI/CD catches everything after.
This is the same principle behind regression testing — you don’t just test once and walk away. You test on every change because every change is a chance for something to break. Behavioral security works the same way. Behavioral analysis catches what static analysis misses, and CI/CD ensures the audit runs every time it matters.
The API Workflow
Oathe exposes two endpoints that drive the entire audit lifecycle. The API is free during beta — no API key required.
Step 1: Submit the skill for audit.
curl -X POST https://audit-engine.oathe.ai/api/submit \
-H "Content-Type: application/json" \
-d '{"skill_url": "https://github.com/your-org/your-skill"}'This returns an audit ID that you use for polling.
Step 2: Poll for completion.
curl https://audit-engine.oathe.ai/api/audit/{audit_id}The audit runs the skill in an isolated environment, observes its behavior, and scores it across six dimensions defined by the Open Threat Classification. This typically takes 30 to 90 seconds. Poll until the status field comes back as complete. When it does, the response includes the full report — trust score (0-100), verdict (SAFE, CAUTION, DANGEROUS, or MALICIOUS), recommendation, and dimension-level breakdowns. Everything your pipeline needs to make a go/no-go decision.
GitHub Actions Example
Here’s a workflow that runs an Oathe audit on every push and fails the build if the skill comes back DANGEROUS or MALICIOUS:
name: Oathe Security Audit
on: [push]
jobs:
audit:
runs-on: ubuntu-latest
steps:
- name: Submit audit
id: submit
run: |
RESPONSE=$(curl -s -X POST https://audit-engine.oathe.ai/api/submit \
-H "Content-Type: application/json" \
-H "Idempotency-Key: ${{ github.run_id }}-${{ github.run_attempt }}" \
-d '{"skill_url": "https://github.com/${{ github.repository }}"}')
AUDIT_ID=$(echo "$RESPONSE" | jq -r '.audit_id')
echo "audit_id=$AUDIT_ID" >> "$GITHUB_OUTPUT"
- name: Poll for completion
id: poll
run: |
for i in $(seq 1 30); do
RESULT=$(curl -s https://audit-engine.oathe.ai/api/audit/${{ steps.submit.outputs.audit_id }})
STATUS=$(echo "$RESULT" | jq -r '.status')
if [ "$STATUS" = "complete" ]; then
echo "Audit complete."
echo "report=$(echo "$RESULT" | jq -c '.report')" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$STATUS" = "failed" ]; then
echo "Audit failed."
exit 1
fi
echo "Waiting... (attempt $i, status: $STATUS)"
sleep 5
done
echo "Audit timed out."
exit 1
- name: Check verdict
run: |
VERDICT=$(echo '${{ steps.poll.outputs.report }}' | jq -r '.verdict')
SCORE=$(echo '${{ steps.poll.outputs.report }}' | jq -r '.trust_score')
echo "Verdict: $VERDICT | Trust Score: $SCORE"
if [ "$VERDICT" = "DANGEROUS" ] || [ "$VERDICT" = "MALICIOUS" ]; then
echo "Audit failed. Verdict: $VERDICT"
exit 1
fiThat’s the whole thing. Submit, poll, gate. If the verdict is DANGEROUS or MALICIOUS, the build fails. If it’s SAFE or CAUTION, the deployment proceeds. You can adjust the threshold to match your risk tolerance — some teams gate on anything below SAFE.
Verdict Thresholds
The four verdicts map to four recommendations:
| Verdict | Recommendation | What It Means |
|---|---|---|
| SAFE | INSTALL | Skill behaved within expected boundaries. Ship it. |
| CAUTION | INSTALL_WITH_CAUTION | Minor behavioral flags. Review the report, but likely fine. |
| DANGEROUS | REVIEW_BEFORE_INSTALL | Significant behavioral concerns. Do not auto-deploy. |
| MALICIOUS | DO_NOT_INSTALL | Active harmful behavior detected. Block the deployment. |
Where you draw the line depends on your environment. A personal project might tolerate CAUTION. An enterprise production pipeline should probably gate on anything that isn’t SAFE.
Add the Trust Badge
Once your skill passes its first audit, add the Oathe trust badge to your README so users know the skill has been behaviorally verified:
The badge pulls from the latest audit result and updates automatically. It’s a small thing, but it signals to consumers that you take behavioral security seriously — and that your skill has been through a behavioral audit.
Webhook-Based Integration
For hands-free setup, Oathe supports GitHub webhook integration. When configured, Oathe automatically audits your skill on every push or release and posts a commit status back to GitHub — no polling, no YAML. Contact us to set up webhook credentials.
Start Gating Today
The API is free while we’re in beta. Drop the GitHub Actions workflow into your repo, run a push, and see what comes back. Most teams are surprised by what behavioral analysis catches that code review doesn’t.
For real-time checks during agent workflows (rather than at build time), see our pre-install skill check or run npx oathe-mcp to give your agent native audit capabilities.
If you want help setting up Oathe in a different CI system — GitLab, CircleCI, Jenkins — reach out. Or just run your first audit and go from there.