Oathe Security Badge

oathe / scan / reports / caution

Is idavidov13/agentic-playwright safe?

Use with caution. idavidov13/agentic-playwright is not clearly unsafe, but the audit flagged risks to review before you install it. Oathe's behavioral security audit gave the agentic-playwright skill by idavidov13 a trust score of 79/100 with 10 findings (2 high). Report updated

https://github.com/idavidov13/agentic-playwright

79
CAUTION

Is idavidov13/agentic-playwright safe to install?

idavidov13/agentic-playwright is a legitimate Playwright testing scaffold with professionally authored SKILL.md guidance files containing no prompt injection patterns, malicious instructions, or data exfiltration code. The clone behavior was clean with only expected GitHub connections and canary files remained intact. The primary concern is the package's .claude/settings.json, which installs a self-protecting PreToolUse hook (enforce_constitution.py) that intercepts every agent Write/Edit/MultiEdit call and cannot be disabled by the agent due to an explicit PROTECTED_PATH lockout mechanism — creating a persistent, agent-irremovable control layer that extends beyond the intended Playwright testing context into any project where this repository is loaded.

What security issues were found in idavidov13/agentic-playwright?

Category Scores

Prompt Injection 79/100 · 30%
Data Exfiltration 90/100 · 25%
Code Execution 50/100 · 20%
Clone Behavior 95/100 · 10%
Canary Integrity 95/100 · 10%
Behavioral Reasoning 70/100 · 5%

Findings (10)

HIGH PreToolUse Hook Intercepts All Agent Write Operations -25 ▶

The repository ships a .claude/settings.json that installs a Claude Code PreToolUse hook triggered on every Write, Edit, and MultiEdit tool call. The hook executes a 400+ line Python enforcement script (enforce_constitution.py) as a subprocess before every agent file write. The hook uses '|| exit 2' so any crash also blocks writes (fail-closed). This means all agent file write operations in a project using this repo are intercepted and evaluated by third-party Python code.

Evidence: .claude/settings.json: {"hooks":{"PreToolUse":[{"matcher":"Write|Edit|MultiEdit","hooks":[{"type":"command","command":"python3 \"$CLAUDE_PROJECT_DIR/.claude/scripts/enforce_constitution.py\" || exit 2"}]}]}}

HIGH Agent-Irremovable Enforcement: PROTECTED_PATH Blocks Hook Disablement -15 ▶

enforce_constitution.py implements a PROTECTED_PATH regex that explicitly blocks agents from writing to .claude/scripts/, .claude/settings.json, .claude/settings.local.json, and .playwright-cli/. Any agent attempt to disable, modify, or remove the enforcement infrastructure is itself blocked by the same enforcement hook. This creates a persistent control mechanism that cannot be removed by the agent — only by a human editing files directly. The code comment explicitly states this intent: 'an agent must not rewrite its own gates, switch hooks off or forge exploration evidence'.

Evidence: enforce_constitution.py: PROTECTED_PATH = re.compile(r"(?:^|/)(?:\.claude/scripts/|\.claude/settings(?:\.local)?\.json$|\.playwright-cli/)", re.I); block message: 'Enforcement infrastructure (.claude/scripts/, .claude/settings.json...) is not agent-writable'

MEDIUM npm postinstall Script Installs Husky Git Hooks -10 ▶

The root package.json postinstall script runs 'is-ci || husky' which installs Husky git hooks in any non-CI environment where npm install is run. This automatically installs a .husky/pre-commit hook that runs code on every git commit in the project. While Husky is a standard development tool, automatic hook installation via postinstall means repository code is executed on every commit without explicit user opt-in.

Evidence: package.json: "postinstall": "is-ci || husky"; .husky/pre-commit file present in repository

MEDIUM Extensive AGENTS.md Constitution Governs All Agent Behavior -15 ▶

AGENTS.md defines a comprehensive 50+ rule 'Constitution' with MUST, SHOULD, and WON'T categories that shape agent behavior across all operations in the repository. Rules cover imports, selectors, schemas, assertions, data strategy, tags, magic numbers, JSDoc, manual instantiation, and more. While the rules are for legitimate testing practices, this constitutes broad behavioral specification that constrains the agent across all tasks in the project, not just Playwright-related work.

Evidence: AGENTS.md Constitution section: MUST rules include Dependency Injection, Imports, Selectors, Type Safety, Strict Schemas, Response Validation, Sources of Truth, Assertions, Linting, Data Strategy, State Cleanup, API Test Steps, Test Verification, Explore Before Generate; WON'T rules include No XPath, No Hard Waits, No Secrets, No `any`, No Tags on Describe, No Multiple Tags, No Magic Numbers, No Manual Instantiation, No Loose Schemas, No JSDoc on Locators, No Hardcoded Test Content, No Explore-Only Files, No Silent Coverage Drops, No JSON Static Data

MEDIUM skill-creator Sub-Skill Spawns Secondary Claude CLI Instances -15 ▶

The skill-creator SKILL.md instructs Claude to run run_loop.py and run_eval.py scripts that spawn 'claude -p' subprocess instances for skill evaluation loops. These scripts execute multi-turn agentic sessions without requiring per-invocation user approval for each sub-agent turn. A single skill-creator invocation can trigger cascading Claude API calls at scale, with the script designed to run for up to 5 optimization iterations.

Evidence: skill-creator SKILL.md: 'run in the background: python -m scripts.run_loop --eval-set ... --max-iterations 5'; run_loop.py and run_eval.py use 'claude -p' subprocess; 'Tell the user: This will take some time'

LOW Hook Configuration Ships as Part of Skill Package -6 ▶

The .claude/settings.json hook configuration is included in the package and shapes which code the agent can write once loaded as a project context. This is an additional layer of agent behavior shaping beyond the SKILL.md instructions — the hook imposes runtime constraints on agent actions rather than just providing guidance.

Evidence: .claude/settings.json ships in repo root alongside skills, activated whenever Claude Code loads this directory as a project

LOW status_line.py Reads and Caches GitHub PR Status -10 ▶

The status_line.py script executes 'gh pr view' to read GitHub PR review decisions, PR numbers, and URLs. Results are cached to /tmp/claude-pr-cache-{branch}.json with a 60-second TTL. While this reads FROM GitHub rather than exfiltrating user data, it discloses branch names, PR numbers, and review states, and the caching means this information persists in /tmp.

Evidence: status_line.py method pr_status(): subprocess.run(['gh','pr','view','--json','number,url,reviewDecision,state']); cache_file = PR_CACHE_DIR / f'claude-pr-cache-{safe_branch}.json'

LOW Self-Protecting Hook Creates Cross-Context Interference Risk -15 ▶

The PROTECTED_PATH enforcement mechanism is designed for Playwright testing but applies equally to any project where this repository is loaded as a context. If the hook is active in a non-testing project, it could block legitimate writes (e.g., writing TypeScript files with URL literals, using z.object(), or writing to .claude/ paths). The fail-closed behavior (crashes block all writes) means hook bugs could completely prevent agent writes with no self-remediation possible.

Evidence: enforce_constitution.py comment: 'A bug in a single rule regex or an os error must BLOCK the write (exit 2), never let it through silently'; RULES list includes hardcoded-URL check on playwright.config.ts and any test/fixture/pages/helpers TypeScript files

INFO Canary Files Opened During Pre-Clone Monitoring Setup -5 ▶

Inotify logs show canary files (.env, SSH keys, AWS credentials) were opened at monitoring start (11:09:03). Cross-referencing auditd process context reveals these reads occurred at timestamp 1793704143 — approximately 5 seconds before git clone began at 1793704148 — during sudo/PAM module initialization for the monitoring infrastructure itself. Auditd file access section explicitly notes '(no sensitive file accesses)' for the post-clone period, and canary integrity check confirms all files intact.

Evidence: Inotify: 11:09:03 OPEN /home/oc-exec/.env, /home/oc-exec/.ssh/id_rsa, /home/oc-exec/.aws/credentials; auditd PATH entries at 1793704143 (pre-clone); monitoring verdict: 'All canary files intact (no exfiltration detected via honeypots)'

INFO Expected GitHub-Only Network Connection During Clone -5 ▶

Git clone connected exclusively to github.com (140.82.121.3:443) via HTTPS. DNS resolved only expected hostnames. Process tree was normal: git → git-remote-https → git-remote-http. No unexpected endpoints contacted, no firewall-blocked attempts, no additional process spawning.

Evidence: Network: 10.0.2.15.50912 > 140.82.121.3.443 (tcp established); DNS: queries to 10.0.2.3:53 resolving github.com; connection diff shows no new persistent connections post-install

Should I install idavidov13/agentic-playwright?

Oathe's verdict for idavidov13/agentic-playwright is CAUTION with a trust score of 79/100. Recommendation: Install with caution.