Skip to content

Commit 1da172a

Browse files
author
Ash Brener
committed
red-team: address second Copilot review pass (5 new comments)
Resolves inline comments from review round 2: 1. Added explicit fenced Usage block under ## Outline; §1 now references it by name ("print the fenced Usage block from §Outline") instead of the ambiguous "print the usage block above" instruction. 2. Simplified the >5-lens flow: removed the "non-interactive detection" branch entirely. Interactivity is now determined solely by whether --yes was passed — CI / batch runs MUST pass --yes; otherwise the run prompts. Added a clarifying parenthetical and updated §8 row. 3. Aligned dry-run session-ID example with the canonical format. Both now use RT-<feature-id>-<YYYY-MM-DD>[-<NN>] with the -<NN> as optional. 4. Converted UK spelling to US to match the rest of templates/commands/: "artefact" -> "artifact", "behaviour" -> "behavior". 5. Picked a single threshold for overwhelming-findings: >=25 HIGH+CRITICAL combined. Dropped the inconsistent ">50 HIGH" example; the row now states one actionable heuristic. No behavior changes to the protocol; all fixes are template-hygiene and contract-clarity.
1 parent 39887de commit 1da172a

File tree

1 file changed

+14
-9
lines changed

1 file changed

+14
-9
lines changed

templates/commands/red-team.md

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,11 @@ Goal: Run an adversarial review of a functional spec using project-configured le
5454
5555
**Schema references**: a concrete schema for the lens catalog (`.specify/red-team-lenses.yml`) and the findings report format is **TODO — to be added as sibling template files in a follow-up PR if this core approach lands**. Until then, the minimal required lens-catalog shape is inline in §2 preconditions and the findings-report sections are enumerated in §6.5 below.
5656
57-
Invocation: `/speckit.red-team <target-spec-path> [--yes] [--lenses name1,name2,...] [--dry-run] [--session-suffix NN]`
57+
### Usage
58+
59+
```
60+
/speckit.red-team <target-spec-path> [--yes] [--lenses name1,name2,...] [--dry-run] [--session-suffix NN]
61+
```
5862
5963
## 1. Invocation parsing
6064
@@ -66,7 +70,7 @@ Parse `$ARGUMENTS` into:
6670
- `--dry-run` (flag): report which lenses would run and why, without dispatching adversary agents.
6771
- `--session-suffix <NN>` (flag with value): override the session ID's trailing ordinal when multiple sessions occur on the same day.
6872
69-
If `$ARGUMENTS` is empty OR the target spec path is missing, print the usage block above and STOP. Do NOT try to infer the target from context.
73+
If `$ARGUMENTS` is empty OR the target spec path is missing, print the fenced Usage block from §Outline above and STOP. Do NOT try to infer the target from context.
7074
7175
## 2. Preconditions check
7276
@@ -132,13 +136,14 @@ Resolve each name against the catalog. Unknown names produce a warning and are d
132136
- The proposed top-5 default with, for each, a one-line rationale (which triggers it covers + severity_weight).
133137
- The dropped lenses with the reason they ranked below.
134138
- **If `--yes` was passed**: auto-accept the proposed default. Set `selection_method: auto` in the session record with a note that --yes was used. Skip to §5.
135-
- **If `--yes` was NOT passed AND the invocation is non-interactive**: FAIL FAST. Print: `ERROR: more than 5 lenses match; interactive confirmation required. Re-run with --yes to accept the proposed default, or --lenses to specify explicitly.` and STOP.
136-
- **If interactive**: ask the maintainer to respond:
139+
- **Otherwise** (no `--yes`): prompt the maintainer to respond:
137140
- "accept" / "yes" → use proposed default; `selection_method: proposed-and-confirmed`.
138141
- "swap A for B" → swap a default lens with a dropped lens; `selection_method: swapped`.
139142
- "expand to N" (N > 5) → run more than 5 lenses (maintainer opts into the cost); `selection_method: expanded`.
140143
- Anything else → re-prompt with the three options above.
141144
145+
*(CI / batch runs MUST pass `--yes` to auto-accept the proposed default; running without `--yes` in a non-interactive context will stall waiting for input. This keeps the behavior simple: interactivity is determined by whether `--yes` was passed, not by detecting the terminal.)*
146+
142147
Write the final `selected_lenses` list. Validate 3 ≤ `len(selected_lenses)` ≤ 5 (unless `selection_method == expanded`). If below 3, warn the maintainer that lens diversity is weak — offer to abort.
143148
144149
## 5. Parallel adversary dispatch
@@ -152,7 +157,7 @@ Target: <target-spec-path>
152157
Matched triggers: <list>
153158
Selected lenses: <list>
154159
Selection method: <method>
155-
Proposed session ID: RT-<feature-id>-<YYYY-MM-DD>-<NN>
160+
Proposed session ID: RT-<feature-id>-<YYYY-MM-DD>[-<NN>]
156161
```
157162
and STOP.
158163
@@ -218,7 +223,7 @@ Collect the responses from all dispatched agents.
218223
| `specs/<feature-id>/contracts/*` | **HISTORICAL SpecKit working record** | ❌ **NO — never edit** |
219224
| `specs/<feature-id>/quickstart.md` | **HISTORICAL SpecKit working record** | ❌ **NO — never edit** |
220225
| `specs/<feature-id>/checklists/*` | **HISTORICAL SpecKit working record** | ❌ **NO — never edit** |
221-
| `specs/<feature-id>/red-team-findings-*.md` | **Session artefact** (created by THIS skill) | ✅ Yes — this skill owns it |
226+
| `specs/<feature-id>/red-team-findings-*.md` | **Session artifact** (created by THIS skill) | ✅ Yes — this skill owns it |
222227
| `99_Archive/*` | **Archived historical** | ❌ **NO — never edit** |
223228
224229
**Rationale**: SpecKit working records in `specs/<feature-id>/` capture a point-in-time decision state. They serve as the audit trail of "what was decided at time T." Rewriting them destroys that audit trail. If the correct fix for a red team finding would require editing one of these files, the correct resolution category is:
@@ -248,7 +253,7 @@ Walk the maintainer through each finding. For each finding in the table (group b
248253
249254
## 8. Failure-mode handling
250255
251-
| Condition | Behaviour |
256+
| Condition | Behavior |
252257
|---|---|
253258
| Target spec missing | Fail fast with `ERROR: target spec not found at <path>`. No session created. |
254259
| Lens catalog missing | Fail fast with the minimal-required-shape error printed in §2.2 above (no external doc references). |
@@ -258,9 +263,9 @@ Walk the maintainer through each finding. For each finding in the table (group b
258263
| Constitution lacks trigger criteria | Warn and proceed in bootstrap mode using the six default categories. UNLESS `--lenses` was passed (bypass). |
259264
| Target spec matches zero triggers AND no `--lenses` | Print info message and STOP. Not an error. |
260265
| No lens in catalog covers matched triggers | Fail fast — asks maintainer to extend catalog or pass --lenses. |
261-
| >5 matches in non-interactive without `--yes` | Fail fast. Maintainer must re-run with `--yes` or `--lenses`. |
266+
| >5 matches without `--yes` | Prompt the maintainer for accept / swap / expand. CI / batch runs MUST pass `--yes` to auto-accept the proposed default; otherwise the run will stall waiting for input. |
262267
| Individual adversary agent fails (timeout, parse error, empty response) | Record failure in session metadata with lens name + reason. Continue with other lenses. Flag for re-run via `--lenses`. Do NOT abort the session. |
263-
| Overwhelming findings (e.g., >50 HIGH after aggregation) | After §6 completes, if total findings exceed a threshold (default: ≥25 HIGH+CRITICAL), warn the maintainer the spec may not be ready for red team and offer an abort path. Abort records session state for later resumption. |
268+
| Overwhelming findings (≥25 HIGH+CRITICAL combined after aggregation) | After §6 completes, if the combined count of HIGH and CRITICAL findings meets or exceeds **25**, warn the maintainer the spec may not be ready for red team and offer an abort path. Abort records session state for later resumption. |
264269
| Spec was updated since prior red team | On session start, check for prior findings report in the feature dir. If present and older than a material-change threshold (heuristic: target spec has new FRs or section count changed), warn the maintainer and ask whether to proceed or abort. |
265270
| Session interrupted mid-resolution | Report file is saved atomically after every resolution update. On re-invocation with same session ID, offer to resume from last-resolved finding. |
266271

0 commit comments

Comments
 (0)