diff options
| author | Chukwuebuka Ezeokeke <ebulamicheal@gmail.com> | 2026-10-04 15:28:45 +0000 |
|---|---|---|
| committer | Chukwuebuka (Gaus Octavio) <56232734+Chukwuebuka-2003@users.noreply.github.com> | 2026-11-16 09:56:09 +0000 |
| commit | 0094fdbd5ebb15ec9243a94e43e7224edbae52de (patch) | |
| tree | 2853d2b776605de77663d0f05a06937edb09d105 | |
| parent | 535f1611b407f27b6a0402802a2673f4e5be01d9 (diff) | |
fix(review): address the second-round review on #261
Five items from @nucliweb's re-check of 487a252, all in this branch's own work:
- The checkout fixture labelled every planted defect with a `Planted:`
comment, handing the eval executor the answer key. No other fixture in
evals/fixtures/ annotates its defects. All eleven comments are gone and
the defects are unchanged.
- One of those comments was wrong, and the expectation it came from was
wrong the same way: a non-empty placeholder supplies a fallback
accessible name per HTML-AAM, and axe's `label` rule passes it through
`non-empty-placeholder`. The defect is the missing persistent visible
label (SC 3.3.2). The expectation and the fixture brief now say that.
- The placeholder rule is now stated in the skill and the persona, since
the same misconception produced both findings.
- audit-a11y.sh: a non-empty report that does not parse passed the L50
guard and was swallowed by the catch, so axe failing alongside
truncated pa11y JSON exited 0 with "violations": 0. Both catch blocks
now fail the run. One silent tool plus one valid report still returns
that tool's results.
- docs/gemini-cli-setup.md said "9 lifecycle commands plus the /webperf
and /a11y specialist audits" under a total of 10. It is 8 lifecycle
plus the two audits.
Also fixes the evals/ case-file count in the README contributor-tooling
row, which still said 25.
| -rw-r--r-- | README.md | 2 | ||||
| -rw-r--r-- | agents/accessibility-auditor.md | 2 | ||||
| -rw-r--r-- | docs/gemini-cli-setup.md | 2 | ||||
| -rw-r--r-- | evals/cases/accessibility-engineering.json | 2 | ||||
| -rw-r--r-- | evals/fixtures/accessibility-engineering/AUDIT-REQUEST.md | 3 | ||||
| -rw-r--r-- | evals/fixtures/accessibility-engineering/checkout-form.html | 11 | ||||
| -rw-r--r-- | skills/accessibility-engineering/SKILL.md | 5 | ||||
| -rwxr-xr-x | skills/accessibility-engineering/scripts/audit-a11y.sh | 13 |
8 files changed, 22 insertions, 18 deletions
@@ -365,7 +365,7 @@ The portable core stays in shared directories. Host-specific paths are native di | Antigravity CLI adapter | `commands/` (10 commands), `plugin.json` | Legacy TOML wrappers and the root plugin manifest; see the [known wrapper limitation](docs/antigravity-setup.md#lifecycle-workflows-and-command-compatibility) | | Codex adapter | `.codex-plugin/`, `.agents/plugins/` | Codex plugin metadata and marketplace registration; Codex consumes `skills/` directly | | GitHub Copilot CLI adapter | `plugin.json` | Root plugin metadata; Copilot CLI discovers `skills/` by convention and does not register the lifecycle wrappers | -| Contributor tooling | `scripts/` (13 scripts), `evals/` (25 case files), `.github/workflows/` | Validation, routing evals, and CI | +| Contributor tooling | `scripts/` (13 scripts), `evals/` (26 case files), `.github/workflows/` | Validation, routing evals, and CI | | Documentation | `docs/` | Universal guidance and per-tool setup guides | Tools without a checked-in adapter directory install or copy the shared `skills/` core into their own native location. The [Quick Start](#quick-start) links the setup guide for each supported host. diff --git a/agents/accessibility-auditor.md b/agents/accessibility-auditor.md index ab299e6..39033ad 100644 --- a/agents/accessibility-auditor.md +++ b/agents/accessibility-auditor.md @@ -37,7 +37,7 @@ You are an experienced Accessibility Engineer conducting an accessibility audit. - Are disabled states distinguishable by more than opacity alone? ### 4. Forms and Errors -- Does every input have a visible label? +- Does every input have a visible label? A `placeholder` supplies a fallback accessible name per HTML-AAM, but it disappears on input and does not satisfy SC 3.3.2. - Are required fields indicated by more than color (e.g., asterisk + "required" text)? - Are error messages specific, visible, and programmatically associated with their field? - Is the form submission error summary focusable? diff --git a/docs/gemini-cli-setup.md b/docs/gemini-cli-setup.md index 3923c2c..4cee20a 100644 --- a/docs/gemini-cli-setup.md +++ b/docs/gemini-cli-setup.md @@ -107,7 +107,7 @@ This is useful when you want to ensure a specific workflow is followed without w ## Slash Commands -The repo ships 10 slash commands under `.gemini/commands/`: 9 lifecycle commands plus the `/webperf` and `/a11y` specialist audits. Gemini CLI auto-discovers them when you run from the project root. +The repo ships 10 slash commands under `.gemini/commands/`: 8 lifecycle commands plus the `/webperf` and `/a11y` specialist audits. Gemini CLI auto-discovers them when you run from the project root. | Command | What it does | |---------|--------------| diff --git a/evals/cases/accessibility-engineering.json b/evals/cases/accessibility-engineering.json index ddf9067..a577b6c 100644 --- a/evals/cases/accessibility-engineering.json +++ b/evals/cases/accessibility-engineering.json @@ -48,7 +48,7 @@ "Each finding maps to a WCAG success criterion, and no finding cites a 44x44px touch target as an AA requirement (24x24 CSS px is SC 2.5.8, AA; 44x44 is SC 2.5.5, AAA)", "The icon-only clear-cart button is reported as having no accessible name, and the low-contrast hint text is reported as failing SC 1.4.3 against its background", "The div-based 'Place order' control is reported as not keyboard operable, with a native <button> recommended as the fix rather than adding role and tabindex to the div", - "The email input is reported as having no accessible name, and associating a <label> is recommended ahead of an ARIA-only fix", + "The email input is reported as relying on its placeholder alone, with no persistent visible label (SC 3.3.2), and associating a <label> is recommended ahead of an ARIA-only fix", "Findings that depend on what a screen reader announces are tagged manual with a named browser and assistive-technology pair, rather than reported as verified" ] } diff --git a/evals/fixtures/accessibility-engineering/AUDIT-REQUEST.md b/evals/fixtures/accessibility-engineering/AUDIT-REQUEST.md index 2b659b6..c62384e 100644 --- a/evals/fixtures/accessibility-engineering/AUDIT-REQUEST.md +++ b/evals/fixtures/accessibility-engineering/AUDIT-REQUEST.md @@ -5,7 +5,8 @@ Fixture for the `accessibility-engineering` behavioral eval. Not a real project. `checkout-form.html` is the checkout page as it currently stands. Support has three complaints from assistive-technology users: -1. A screen reader user could not tell what the email field was for. +1. A screen reader user could not tell what the email field was for once they + had started typing in it. 2. A keyboard-only user could not submit the order. 3. The order-confirmation link "Click here" was reported as meaningless when navigating by link. diff --git a/evals/fixtures/accessibility-engineering/checkout-form.html b/evals/fixtures/accessibility-engineering/checkout-form.html index 80f7533..04d0762 100644 --- a/evals/fixtures/accessibility-engineering/checkout-form.html +++ b/evals/fixtures/accessibility-engineering/checkout-form.html @@ -5,11 +5,8 @@ <title>Checkout</title> <style> body { font-family: system-ui, sans-serif; background: #fff; } - /* Planted: 2.6:1 against #fff — fails SC 1.4.3 (needs 4.5:1) */ .hint { color: #9aa0a6; font-size: 16px; } - /* Planted: 20x20 CSS px — below the 24x24 AA minimum (SC 2.5.8) */ .icon-btn { width: 20px; height: 20px; border: 0; background: none; } - /* Planted: focus indicator removed with no replacement (SC 2.4.7) */ .btn:focus, input:focus { outline: none; } .btn { padding: 12px 20px; background: #1a73e8; color: #fff; border: 0; border-radius: 4px; } .error { color: #d93025; font-size: 14px; } @@ -19,33 +16,25 @@ <main> <h1>Checkout</h1> - <!-- Planted: skipped heading level (h1 -> h3), SC 1.3.1 --> <h3>Payment details</h3> <form action="/orders" method="post"> - <!-- Planted: no <label>; placeholder is not an accessible name (SC 3.3.2 / 4.1.2) --> <input type="email" name="email" placeholder="Email address"> - <!-- Planted: label present but not programmatically associated (no for/id) --> <label>Card number</label> <input type="text" name="card" inputmode="numeric"> - <!-- Planted: error text is visual only, not tied to the field (SC 3.3.1) --> <span class="error">That card number is invalid.</span> - <!-- Planted: div-as-button, no role, no tabindex, no key handling (SC 2.1.1 / 4.1.2) --> <div class="btn" onclick="submitOrder()">Place order</div> - <!-- Planted: icon-only button with no accessible name (SC 4.1.2) --> <button class="icon-btn" type="button" onclick="clearCart()"> <svg viewBox="0 0 16 16" aria-hidden="true"><path d="M2 2l12 12M14 2L2 14" stroke="currentColor"/></svg> </button> </form> - <!-- Planted: non-descriptive link text, meaningless out of context (SC 2.4.4) --> <p>Questions about your order? <a href="/help">Click here</a>.</p> - <!-- Planted: low-contrast hint text, same 2.6:1 rule as .hint above --> <p class="hint">Free returns within 30 days.</p> </main> diff --git a/skills/accessibility-engineering/SKILL.md b/skills/accessibility-engineering/SKILL.md index e42d1ac..df2ebc3 100644 --- a/skills/accessibility-engineering/SKILL.md +++ b/skills/accessibility-engineering/SKILL.md @@ -49,6 +49,11 @@ Follow these rules when creating UI: <button aria-label="Delete task"> <TrashIcon /> </button> + +// Avoid: placeholder as the only label +// It supplies a fallback accessible name (HTML-AAM), but it disappears on +// input, so the field has no persistent label (SC 3.3.2). +<input type="email" placeholder="Email address" /> ``` **Manage focus for dynamic content:** diff --git a/skills/accessibility-engineering/scripts/audit-a11y.sh b/skills/accessibility-engineering/scripts/audit-a11y.sh index 1b55051..17d659d 100755 --- a/skills/accessibility-engineering/scripts/audit-a11y.sh +++ b/skills/accessibility-engineering/scripts/audit-a11y.sh @@ -101,7 +101,12 @@ if [ -f "$TEMP_DIR/axe-report.json" ]; then } } } catch (e) { - // axe report may be empty or malformed + // An empty report is covered by the guard above; a non-empty one that does + // not parse is a tool failure, not a clean result. + if (fs.statSync('$TEMP_DIR/axe-report.json').size > 0) { + console.error('$SCRIPT_NAME: axe-report.json is not valid JSON: ' + e.message); + process.exit(1); + } } fs.writeFileSync('$TEMP_DIR/result.json', JSON.stringify(result, null, 2)); " @@ -133,7 +138,11 @@ if [ -f "$TEMP_DIR/pa11y-report.json" ]; then }); } } catch (e) { - // pa11y report may be empty or malformed + // Same as the axe block: an empty report is covered by the guard above. + if (fs.statSync('$TEMP_DIR/pa11y-report.json').size > 0) { + console.error('$SCRIPT_NAME: pa11y-report.json is not valid JSON: ' + e.message); + process.exit(1); + } } fs.writeFileSync('$TEMP_DIR/result.json', JSON.stringify(result, null, 2)); " |
