| Age | Commit message (Collapse) | Author |
|
tests/claude-code/test-subagent-driven-development.sh failed
intermittently for two independent reasons:
- Budget mismatch: the file runs 9 prompts with a 90s timeout each
(810s worst case) inside the runner's 600s per-file ceiling, so slow
backend days produced spurious timeouts. Raise the runner default to
900s and fix the help text, which claimed the default was 300.
- Case-sensitive prose matching: the assert helpers grepped free-form
model output case-sensitively, but models capitalize the skill's own
headings — observed failures include "Do Not Trust the Report"
missing pattern "not trust" and a structured answer missing
"First:.*spec.*compliance". Match case-insensitively in
assert_contains/assert_not_contains/assert_count/assert_order, widen
two Test 5 keyword patterns to phrasings observed in real runs, and
make assert_order dump the output on failure the way assert_contains
already does, so the next flake is diagnosable.
Observed 3 failures across 4 runs before the change (timeout, two
distinct pattern misses); 3/3 consecutive full runs pass after it.
|
|
|
|
- test-worktree-native-preference.sh: drill covers PRESSURE phase only;
RED + GREEN baselines have no drill counterpart and are kept so
the RED-GREEN-REFACTOR validation remains rerunnable end-to-end.
- test-subagent-driven-development-integration.sh: drill covers the
YAGNI subset (forbidden exports + reviewer-as-gate). Bash adds
>=3 commits, >=2 subagent dispatches, TodoWrite usage, test file
existence check, and token-budget telemetry. Kept until drill
scenario covers those or they are retired.
- test-subagent-driven-development.sh: tests agent's ability to
*describe* SDD (string matches against expected keywords). Drill
scenarios test behavior, not description-recall. Kept by design.
Subagent verification recorded in commit messages of subsequent
deletions; gap analyses driving these annotations are also in the
verification subagent reports for the gating sweep.
|
|
Instead of prohibiting main branch work entirely, allow it with explicit
user consent. This is more flexible while still ensuring users are aware
of the implications.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|
TDD: Test verifies that subagent-driven-development skill warns
against starting implementation directly on main/master branch.
Test expects skill to recommend worktree or feature branch instead.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|
Add Test 8 to verify that using-git-worktrees is mentioned as a required
skill for subagent-driven-development. This test will initially fail per
TDD approach - the skill file needs to be updated to pass this test.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|
The assertion now matches "subagent-driven-development", "Subagent-Driven
Development", and "Subagent Driven" since Claude's responses may use
different casing and formatting styles.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|
Created automated test suite for testing superpowers skills using
Claude Code CLI in headless mode.
New files:
- tests/claude-code/run-skill-tests.sh - Main test runner
- tests/claude-code/test-helpers.sh - Helper functions for testing
- tests/claude-code/test-subagent-driven-development.sh - First test
- tests/claude-code/README.md - Documentation
Test framework features:
- Run Claude Code with prompts and capture output
- Assertion helpers (contains, not_contains, count, order)
- Test project creation helpers
- Timeout support (default 5 minutes)
- Verbose mode for debugging
- Specific test selection
First test verifies subagent-driven-development skill:
- Skill loading
- Workflow ordering (spec compliance before code quality)
- Self-review requirements
- Plan reading efficiency (read once)
- Spec compliance reviewer skepticism
- Review loops
- Task context provision
Run with: cd tests/claude-code && ./run-skill-tests.sh
|