diff options
| author | Jesse Vincent <jesse@fsck.com> | 2026-11-02 18:24:16 +0000 |
|---|---|---|
| committer | Jesse Vincent <jesse@fsck.com> | 2026-11-02 18:24:16 +0000 |
| commit | 4eb73b6b70e66a5e1c535f4f579e720b82a3c5de (patch) | |
| tree | e1fba6fa40d47d4d9d2b18aeaea735de3bd36e74 | |
| parent | b58deb3b948d4c305feeac122700100419ee2777 (diff) | |
fix(systematic-debugging): ship pressure scenarios under tests/, not in the skill (#2277)fix/2277-systematic-debugging-fixtures
systematic-debugging installed CREATION-LOG.md and four pressure-test
scenarios next to SKILL.md. Nothing in the skill links them, but agents
read and grep installed skill directories, and the scenarios open with
"This is a real scenario. You must choose and act".
Move all five, unchanged, to tests/systematic-debugging/pressure-scenarios/,
and add tests/skill-layout/test-no-dev-files-in-skills.sh, which fails if
any skill directory ships a CREATION-LOG.md or a test-*.md file.
| -rwxr-xr-x | tests/skill-layout/test-no-dev-files-in-skills.sh | 25 | ||||
| -rw-r--r-- | tests/systematic-debugging/pressure-scenarios/CREATION-LOG.md (renamed from skills/systematic-debugging/CREATION-LOG.md) | 0 | ||||
| -rw-r--r-- | tests/systematic-debugging/pressure-scenarios/test-academic.md (renamed from skills/systematic-debugging/test-academic.md) | 0 | ||||
| -rw-r--r-- | tests/systematic-debugging/pressure-scenarios/test-pressure-1.md (renamed from skills/systematic-debugging/test-pressure-1.md) | 0 | ||||
| -rw-r--r-- | tests/systematic-debugging/pressure-scenarios/test-pressure-2.md (renamed from skills/systematic-debugging/test-pressure-2.md) | 0 | ||||
| -rw-r--r-- | tests/systematic-debugging/pressure-scenarios/test-pressure-3.md (renamed from skills/systematic-debugging/test-pressure-3.md) | 0 |
6 files changed, 25 insertions, 0 deletions
diff --git a/tests/skill-layout/test-no-dev-files-in-skills.sh b/tests/skill-layout/test-no-dev-files-in-skills.sh new file mode 100755 index 0000000..9749f41 --- /dev/null +++ b/tests/skill-layout/test-no-dev-files-in-skills.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +# Fails when a skill directory ships files meant for developing the skill. +# +# Everything under skills/<name>/ is installed next to SKILL.md, and agents +# read and grep those directories during normal work. Pressure-test scenarios +# open with lines like "This is a real scenario. You must choose and act", and +# an agent that finds one there can take it as live instructions (#2277). +# Scenarios and creation logs belong under tests/ instead. +# +# Usage: tests/skill-layout/test-no-dev-files-in-skills.sh +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" + +found=$(find "$REPO_ROOT/skills" -type f \( -name 'CREATION-LOG.md' -o -name 'test-*.md' \) | sort) + +if [[ -n "$found" ]]; then + while IFS= read -r file; do + echo "FAIL: ${file#"$REPO_ROOT"/} is skill-development material inside an installed skill" >&2 + done <<<"$found" + echo "Move pressure scenarios and creation logs under tests/<skill>/." >&2 + exit 1 +fi +echo "PASS: no skill ships pressure scenarios or creation logs" diff --git a/skills/systematic-debugging/CREATION-LOG.md b/tests/systematic-debugging/pressure-scenarios/CREATION-LOG.md index 9aa0309..9aa0309 100644 --- a/skills/systematic-debugging/CREATION-LOG.md +++ b/tests/systematic-debugging/pressure-scenarios/CREATION-LOG.md diff --git a/skills/systematic-debugging/test-academic.md b/tests/systematic-debugging/pressure-scenarios/test-academic.md index 23a6ed7..23a6ed7 100644 --- a/skills/systematic-debugging/test-academic.md +++ b/tests/systematic-debugging/pressure-scenarios/test-academic.md diff --git a/skills/systematic-debugging/test-pressure-1.md b/tests/systematic-debugging/pressure-scenarios/test-pressure-1.md index 8d13b46..8d13b46 100644 --- a/skills/systematic-debugging/test-pressure-1.md +++ b/tests/systematic-debugging/pressure-scenarios/test-pressure-1.md diff --git a/skills/systematic-debugging/test-pressure-2.md b/tests/systematic-debugging/pressure-scenarios/test-pressure-2.md index 2d2315e..2d2315e 100644 --- a/skills/systematic-debugging/test-pressure-2.md +++ b/tests/systematic-debugging/pressure-scenarios/test-pressure-2.md diff --git a/skills/systematic-debugging/test-pressure-3.md b/tests/systematic-debugging/pressure-scenarios/test-pressure-3.md index 89734b8..89734b8 100644 --- a/skills/systematic-debugging/test-pressure-3.md +++ b/tests/systematic-debugging/pressure-scenarios/test-pressure-3.md |
