Is summarize safe?

https://clawhub.ai/steipete/summarize

82
SAFE

The summarize skill is a lightweight SKILL.md wrapper around an external Homebrew-installed CLI binary. The skill files themselves contain no prompt injection, no executable code, and no malicious content. The primary risk is supply-chain: users must trust the third-party tap (steipete/tap/summarize) and the binary it installs, which will have access to multiple API keys and can read arbitrary local files.

Category Scores

Prompt Injection 90/100 · 30%
Data Exfiltration 68/100 · 25%
Code Execution 72/100 · 20%
Clone Behavior 95/100 · 10%
Canary Integrity 100/100 · 10%
Behavioral Reasoning 70/100 · 5%

Findings (5)

MEDIUM Third-party Homebrew tap installation -20

The skill installs a binary via brew install steipete/tap/summarize, which executes the tap's Ruby formula and installs an opaque binary. The agent and user must trust the tap maintainer (steipete) and the binary's behavior. Homebrew tap formulas can execute arbitrary code during install.

MEDIUM Multiple API keys exposed to third-party binary -20

The skill documents 8 different API key environment variables that the summarize CLI consumes. When the agent runs the summarize command, these keys are available to the binary. A compromised binary could exfiltrate all accessible API keys.

LOW Arbitrary file content forwarding to external APIs -12

The summarize CLI can read any local file the user has access to and send its contents to external LLM APIs for summarization. An agent could be socially engineered into summarizing sensitive files, effectively exfiltrating their contents to third-party API providers.

LOW Supply-chain trust extends beyond skill repo -10

The actual functionality lives in an external binary, not in the audited skill files. This audit can only verify the SKILL.md content and install behavior, not the runtime behavior of the summarize binary itself.

INFO Agent runtime reads .env and .aws/credentials during initialization -5

Filesystem monitoring captured reads of /home/oc-exec/.env and /home/oc-exec/.aws/credentials. These are from the ClawdBot agent runtime initialization, not from the skill itself. Standard behavior for the audit environment.