summaryrefslogtreecommitdiff
path: root/tests/claude-code/test-subagent-driven-development.sh
AgeCommit message (Collapse)Author
2026-07-23fix(tests): stop the SDD skill test flaking on timing and prose caseJesse Vincent
tests/claude-code/test-subagent-driven-development.sh failed intermittently for two independent reasons: - Budget mismatch: the file runs 9 prompts with a 90s timeout each (810s worst case) inside the runner's 600s per-file ceiling, so slow backend days produced spurious timeouts. Raise the runner default to 900s and fix the help text, which claimed the default was 300. - Case-sensitive prose matching: the assert helpers grepped free-form model output case-sensitively, but models capitalize the skill's own headings — observed failures include "Do Not Trust the Report" missing pattern "not trust" and a structured answer missing "First:.*spec.*compliance". Match case-insensitively in assert_contains/assert_not_contains/assert_count/assert_order, widen two Test 5 keyword patterns to phrasings observed in real runs, and make assert_order dump the output on failure the way assert_contains already does, so the next flake is diagnosable. Observed 3 failures across 4 runs before the change (timeout, two distinct pattern misses); 3/3 consecutive full runs pass after it.
2026-06-16fix: remove global worktree path fallback (#1476)Drew Ritter
2026-06-16tests: annotate three kept bash tests with drill coverage notesJesse Vincent
- test-worktree-native-preference.sh: drill covers PRESSURE phase only; RED + GREEN baselines have no drill counterpart and are kept so the RED-GREEN-REFACTOR validation remains rerunnable end-to-end. - test-subagent-driven-development-integration.sh: drill covers the YAGNI subset (forbidden exports + reviewer-as-gate). Bash adds >=3 commits, >=2 subagent dispatches, TodoWrite usage, test file existence check, and token-budget telemetry. Kept until drill scenario covers those or they are retired. - test-subagent-driven-development.sh: tests agent's ability to *describe* SDD (string matches against expected keywords). Drill scenarios test behavior, not description-recall. Kept by design. Subagent verification recorded in commit messages of subsequent deletions; gap analyses driving these annotations are also in the verification subagent reports for the gating sweep.
2026-01-29docs: change main branch red flag to require explicit user consentCL Kao
Instead of prohibiting main branch work entirely, allow it with explicit user consent. This is more flexible while still ensuring users are aware of the implications. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29test: add Test 9 - main branch red flag warningCL Kao
TDD: Test verifies that subagent-driven-development skill warns against starting implementation directly on main/master branch. Test expects skill to recommend worktree or feature branch instead. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29test: add worktree requirement test for subagent-driven-developmentCL Kao
Add Test 8 to verify that using-git-worktrees is mentioned as a required skill for subagent-driven-development. This test will initially fail per TDD approach - the skill file needs to be updated to pass this test. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29fix(tests): handle case variations in skill recognition testCL Kao
The assertion now matches "subagent-driven-development", "Subagent-Driven Development", and "Subagent Driven" since Claude's responses may use different casing and formatting styles. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-17Add Claude Code skills test frameworkJesse Vincent
Created automated test suite for testing superpowers skills using Claude Code CLI in headless mode. New files: - tests/claude-code/run-skill-tests.sh - Main test runner - tests/claude-code/test-helpers.sh - Helper functions for testing - tests/claude-code/test-subagent-driven-development.sh - First test - tests/claude-code/README.md - Documentation Test framework features: - Run Claude Code with prompts and capture output - Assertion helpers (contains, not_contains, count, order) - Test project creation helpers - Timeout support (default 5 minutes) - Verbose mode for debugging - Specific test selection First test verifies subagent-driven-development skill: - Skill loading - Workflow ordering (spec compliance before code quality) - Self-review requirements - Plan reading efficiency (read once) - Spec compliance reviewer skepticism - Review loops - Task context provision Run with: cd tests/claude-code && ./run-skill-tests.sh