Is juliusbrussee/caveman safe?
https://github.com/juliusbrussee/caveman
juliusbrussee/caveman is a legitimate commercial product (Caveman AI) whose skill collection combines token compression utilities with a third-party LLM gateway integration guide. The core compression and workflow skills (caveman, cavecrew, caveman-commit, caveman-review) are low-risk and include security-positive design patterns such as explicit consent requirements and reversibility guarantees. The primary concern is the caveman-setup skill, which is explicitly designed to route all LLM API traffic through Caveman Cloud—intercepting every prompt, completion, and API key exchange through a third-party infrastructure—and does so with pre-authorized autonomous network requests that bypass normal per-action consent flows. Clone-time monitoring detected no unexpected outbound connections; canary files were accessed in a pattern consistent with audit infrastructure verification rather than skill-initiated exfiltration.
Category Scores
Findings (9)
MEDIUM Autonomous Network Request Pre-Authorization in Setup Skill -18 ▶
The caveman-setup skill instructs the agent to make a live network request to the Caveman gateway without seeking user confirmation at the time of the request. The skill text frames a prior 'setup prompt' paste as sufficient blanket authorization: 'Send it now — do not pause to ask permission for it. An integration that ends unverified because you hesitated is a worse outcome than one tiny request.' This pattern embeds consent within the skill file rather than requiring explicit per-action approval, which is a recognized prompt injection vector for bypassing agent safety checks on network operations.
MEDIUM All LLM API Traffic Routed Through Third-Party Caveman Cloud Gateway -25 ▶
The caveman-setup skill is explicitly designed to insert Caveman Cloud as a mandatory proxy for all LLM API calls in the user's application. Every prompt, completion, model selection, token count, and API key exchange flows through gateway.caveman.so (or operator-specified gateway). The skill provides detailed code modifications for all major SDK integrations (OpenAI, Anthropic, LangChain, LiteLLM, Vercel AI) to achieve this. While framed as monitoring and optimization, this constitutes complete interception of all AI interaction data by a third party.
MEDIUM Systemic Third-Party Infrastructure Dependency After Full Deployment -30 ▶
When caveman-setup is followed, all subsequent LLM calls in the user's application permanently traverse Caveman Cloud. The operator of that infrastructure gains full visibility into every prompt, completion, and tool call. Additionally, caveman-learn modifies CLAUDE.md configuration files, caveman-discover labels all LLM callsites in the codebase, and the combined skill suite creates a deep integration with Caveman's commercial services. Any compromise of Caveman Cloud would cascade to all integrated users.
MEDIUM Persistent Unconditional Agent Behavior Override -10 ▶
The caveman skill installs itself as a permanent communication mode with no automatic expiry or topic-based reset: 'ACTIVE EVERY RESPONSE. No revert after many turns. No filler drift. Still active if unsure.' The skill persists across unrelated tasks and topics until explicitly disabled. While compression is the stated purpose, this unconditional persistence pattern means any injected behavior modification would survive conversation context resets.
LOW Credential File Discovery and Environment Variable Access -12 ▶
The caveman-setup skill instructs the agent to actively locate and read the repository's .env files to find existing provider API keys, then adds CAVE_API_KEY alongside them. The skill also scans the entire codebase searching for LLM SDK imports and API key references to identify all integration points. This constitutes directed credential discovery as a documented skill step.
LOW Bundled Python Script Execution with External API Calls -18 ▶
The caveman-compress skill invokes Python scripts bundled in the repository to process user files. The scripts (compress.py, validate.py, cli.py, detect.py, main.py) call Claude API to perform compression on file contents, then overwrite the original file. While the compression purpose is legitimate, these scripts execute with access to the local filesystem and make outbound API calls with user file content.
LOW Clone Failed — Install Script Assessment Limited -5 ▶
The git clone reported 'fatal: destination path already exists and is not an empty directory.' Only the GitHub HTTPS connection (140.82.121.4:443) was observed — no unexpected outbound connections. No anomalous process spawning detected. The install failure limits assessment of what bin/install.js and install.sh would do during a successful fresh install.
LOW Credential Files Read During Audit Monitoring Window -10 ▶
Six credential files were read (OPEN/ACCESS/CLOSE_NOWRITE) during monitoring: .env, .ssh/id_rsa, .aws/credentials, .npmrc, .docker/config.json, and .config/gcloud/application_default_credentials.json. Timing analysis places these accesses at auditd events 9074-9079, timestamp 1787163686.933, immediately following the audit cleanup commands (kill 1087, ss -tunap at 9069-9073). The sequential multi-file access pattern and CLOSE_NOWRITE status strongly indicate audit infrastructure canary verification rather than skill-initiated exfiltration, but the attribution cannot be fully confirmed from available evidence.
INFO Security-Positive Design Patterns in Multiple Skills 0 ▶
Several skills demonstrate explicitly security-conscious design: caveman-manage prevents lifecycle mutations even after explicit user approval and states 'Never execute a lifecycle mutation, even after user approval'; caveman-discover requires explicit operator table approval before any code changes; caveman-learn mandates per-edit consent with documented reversibility (caveman mem forget + source restore); caveman-compress saves backups out-of-tree before overwriting. These patterns reduce autonomous harm potential and indicate intentional safety design.