Skip to content

Commit 07ef363

Browse files
MDA2AVMDA2AV
authored
Merge pull request #44 from MDA2AV/ui/improvements-ai-agents
Add AI Contribution and RFC Dashboard
2 parents b27034d + 12afcc9 commit 07ef363

11 files changed

Lines changed: 857 additions & 7 deletions

File tree

‎AGENTS.md‎

Lines changed: 295 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,295 @@
1+
# Http11Probe — AI Agent Contribution Guide
2+
3+
This file is designed for LLM/AI agent consumption. It contains precise, unambiguous instructions for adding a new test or a new framework to the Http11Probe platform.
4+
5+
## Project overview
6+
7+
Http11Probe is an HTTP/1.1 compliance and security tester. It sends raw TCP requests to servers and validates responses against RFC 9110/9112. The codebase is C# / .NET 10. Documentation is a Hugo + Hextra static site under `docs/`.
8+
9+
---
10+
11+
## TASK A: Add a new test
12+
13+
Adding a test requires changes to **4 locations** (sometimes 3 if URL mapping is automatic).
14+
15+
### Step 1 — Add the test case to the suite file
16+
17+
Choose the correct suite file based on category:
18+
19+
| Category | File path |
20+
|----------|-----------|
21+
| Compliance | `src/Http11Probe/TestCases/Suites/ComplianceSuite.cs` |
22+
| Smuggling | `src/Http11Probe/TestCases/Suites/SmugglingSuite.cs` |
23+
| Malformed Input | `src/Http11Probe/TestCases/Suites/MalformedInputSuite.cs` |
24+
| Normalization | `src/Http11Probe/TestCases/Suites/NormalizationSuite.cs` |
25+
26+
Append a `yield return new TestCase { ... };` inside the `GetTestCases()` method. Here is the full schema:
27+
28+
```csharp
29+
yield return new TestCase
30+
{
31+
// REQUIRED fields
32+
Id = "COMP-EXAMPLE", // Unique ID. Prefix conventions below.
33+
Description = "What this test checks", // One-line human description.
34+
Category = TestCategory.Compliance, // Compliance | Smuggling | MalformedInput | Normalization
35+
PayloadFactory = ctx => MakeRequest( // Builds the raw HTTP bytes to send.
36+
$"GET / HTTP/1.1\r\nHost: {ctx.HostHeader}\r\n\r\n"
37+
),
38+
Expected = new ExpectedBehavior // How to validate the response. See below.
39+
{
40+
ExpectedStatus = StatusCodeRange.Exact(400),
41+
},
42+
43+
// OPTIONAL fields
44+
RfcReference = "RFC 9112 §5.1", // Use § not "Section". Omit if no RFC applies.
45+
Scored = true, // Default true. Set false for MAY/informational tests.
46+
AllowConnectionClose = false, // On Expected. See validation rules below.
47+
FollowUpPayloadFactory = ctx => ..., // Second request on same connection (pipeline tests).
48+
RequiresConnectionReuse = false, // True if FollowUpPayloadFactory needs same TCP connection.
49+
BehavioralAnalyzer = (response) => ..., // Optional Func<HttpResponse?, string?> for analysis notes.
50+
};
51+
```
52+
53+
**Test ID prefix conventions:**
54+
55+
| Prefix | Suite |
56+
|--------|-------|
57+
| `COMP-` | Compliance |
58+
| `SMUG-` | Smuggling |
59+
| `MAL-` | Malformed Input |
60+
| `NORM-` | Normalization |
61+
| `RFC9112-X.X-` or `RFC9110-X.X-` | Compliance (maps directly to an RFC section) |
62+
63+
**Validation patterns — choose ONE:**
64+
65+
Pattern 1 — Exact status, no alternatives:
66+
```csharp
67+
Expected = new ExpectedBehavior
68+
{
69+
ExpectedStatus = StatusCodeRange.Exact(400),
70+
}
71+
```
72+
Use for strict MUST-400 requirements (e.g. SP-BEFORE-COLON, MISSING-HOST, DUPLICATE-HOST, OBS-FOLD, CR-ONLY).
73+
74+
Pattern 2 — Status with connection close as alternative:
75+
```csharp
76+
Expected = new ExpectedBehavior
77+
{
78+
ExpectedStatus = StatusCodeRange.Exact(400),
79+
AllowConnectionClose = true,
80+
}
81+
```
82+
Use when close is acceptable instead of a status code.
83+
84+
Pattern 3 — Custom validator (takes priority over ExpectedStatus):
85+
```csharp
86+
Expected = new ExpectedBehavior
87+
{
88+
CustomValidator = (response, state) =>
89+
{
90+
if (state == ConnectionState.ClosedByServer && response is null) return TestVerdict.Pass;
91+
if (response is null) return TestVerdict.Fail;
92+
if (response.StatusCode == 400) return TestVerdict.Pass;
93+
if (response.StatusCode >= 200 && response.StatusCode < 300) return TestVerdict.Warn;
94+
return TestVerdict.Fail;
95+
},
96+
Description = "400 or close = pass, 2xx = warn",
97+
}
98+
```
99+
Use for pass/warn/fail logic, timeout acceptance, or multi-outcome tests.
100+
101+
**Available StatusCodeRange factories:**
102+
- `StatusCodeRange.Exact(int code)` — single status code
103+
- `StatusCodeRange.Range(int start, int end)` — inclusive range
104+
- `StatusCodeRange.Range2xx` — 200-299
105+
- `StatusCodeRange.Range4xx` — 400-499
106+
- `StatusCodeRange.Range4xxOr5xx` — 400-599
107+
108+
**Available TestVerdict values:** `Pass`, `Fail`, `Warn`, `Skip`, `Error`
109+
110+
**Available ConnectionState values:** `Open`, `ClosedByServer`, `TimedOut`, `Error`
111+
112+
**Helper method available in all suites:**
113+
```csharp
114+
private static byte[] MakeRequest(string request) => Encoding.ASCII.GetBytes(request);
115+
```
116+
117+
**Critical rules:**
118+
- NEVER set `AllowConnectionClose = true` for MUST-400 requirements where the RFC explicitly says "respond with 400".
119+
- Set `Scored = false` only for MAY-level or purely informational tests.
120+
- Always use `ctx.HostHeader` (not a hardcoded host) in payloads.
121+
- Tests are auto-discovered — no registration step needed. The `GetTestCases()` yield return is sufficient.
122+
123+
### Step 2 — Add docs URL mapping (conditional)
124+
125+
**File:** `src/Http11Probe.Cli/Reporting/DocsUrlMap.cs`
126+
127+
This step is **only needed** for `COMP-*` and `RFC*` prefixed tests. The following prefixes are auto-mapped:
128+
- `SMUG-XYZ` → `smuggling/xyz` (lowercased)
129+
- `MAL-XYZ` → `malformed-input/xyz` (lowercased)
130+
- `NORM-XYZ` → `normalization/xyz` (lowercased)
131+
132+
For compliance tests, add an entry to the `ComplianceSlugs` dictionary:
133+
```csharp
134+
["COMP-EXAMPLE"] = "headers/example",
135+
```
136+
137+
If the doc filename doesn't match the auto-mapping convention, add to `SpecialSlugs` instead:
138+
```csharp
139+
["MAL-CHUNK-EXT-64K"] = "malformed-input/chunk-extension-long",
140+
```
141+
142+
### Step 3 — Create the documentation page
143+
144+
**File:** `docs/content/docs/{category-slug}/{test-slug}.md`
145+
146+
Category slug mapping:
147+
148+
| Category | Slug |
149+
|----------|------|
150+
| Compliance (line endings) | `line-endings` |
151+
| Compliance (request line) | `request-line` |
152+
| Compliance (headers) | `headers` |
153+
| Compliance (host header) | `host-header` |
154+
| Compliance (content-length) | `content-length` |
155+
| Compliance (body) | `body` |
156+
| Compliance (upgrade) | `upgrade` |
157+
| Smuggling | `smuggling` |
158+
| Malformed Input | `malformed-input` |
159+
| Normalization | `normalization` |
160+
161+
Use this exact template:
162+
163+
```markdown
164+
---
165+
title: "EXAMPLE"
166+
description: "EXAMPLE test documentation"
167+
weight: 1
168+
---
169+
170+
| | |
171+
|---|---|
172+
| **Test ID** | `COMP-EXAMPLE` |
173+
| **Category** | Compliance |
174+
| **RFC** | [RFC 9112 §X.X](https://www.rfc-editor.org/rfc/rfc9112#section-X.X) |
175+
| **Requirement** | MUST |
176+
| **Expected** | `400` or close |
177+
178+
## What it sends
179+
180+
A request with [description of the non-conforming element].
181+
182+
\```http
183+
GET / HTTP/1.1\r\n
184+
Host: localhost:8080\r\n
185+
[malformed element]\r\n
186+
\r\n
187+
\```
188+
189+
## What the RFC says
190+
191+
> "Exact quote from the RFC with the MUST/SHOULD/MAY keyword." -- RFC 9112 Section X.X
192+
193+
Explanation of what the quote means for this test.
194+
195+
## Why it matters
196+
197+
Security and compatibility implications. Why this matters for real-world deployments.
198+
199+
## Sources
200+
201+
- [RFC 9112 §X.X](https://www.rfc-editor.org/rfc/rfc9112#section-X.X)
202+
```
203+
204+
**Requirement field values:** `MUST`, `SHOULD`, `MAY`, `"ought to"`, `Implicit MUST (grammar violation)`, `Unscored`, or a descriptive phrase like `MUST reject or replace with SP`.
205+
206+
### Step 4 — Add a card to the category index
207+
208+
**File:** `docs/content/docs/{category-slug}/_index.md`
209+
210+
Find the `{{</* cards */>}}` block and add a new card entry. Place scored tests before unscored tests.
211+
212+
```
213+
{{</* card link="example" title="EXAMPLE" subtitle="Short description of what the test checks." */>}}
214+
```
215+
216+
The `link` value is the filename without `.md`.
217+
218+
### Verification checklist
219+
220+
After making all changes:
221+
222+
1. `dotnet build Http11Probe.slnx -c Release` — must compile without errors.
223+
2. The new test ID appears in the output of `dotnet run --project src/Http11Probe.Cli -- --host localhost --port 8080`.
224+
3. Hugo docs render: `cd docs && hugo server` — the new page is accessible and linked from its category index.
225+
226+
---
227+
228+
## TASK B: Add a new framework
229+
230+
Adding a framework requires creating **3 files** in a new directory. No existing files need modification.
231+
232+
### Step 1 — Create the server directory
233+
234+
Create a new directory: `src/Servers/YourServer/`
235+
236+
### Step 2 — Implement the server
237+
238+
Your server MUST listen on **port 8080** and implement these endpoints:
239+
240+
| Endpoint | Method | Behavior |
241+
|----------|--------|----------|
242+
| `/` | `GET` | Return `200 OK` |
243+
| `/` | `HEAD` | Return `200 OK` with no body |
244+
| `/` | `POST` | Read the full request body and return it in the response body |
245+
| `/` | `OPTIONS` | Return `200 OK` |
246+
| `/echo` | `POST` | Return all received request headers in the response body, one per line as `Name: Value` |
247+
248+
The `/echo` endpoint is critical for normalization tests. It must echo back all headers the server received, preserving the names as the server internally represents them.
249+
250+
Example `/echo` response body:
251+
```
252+
Host: localhost:8080
253+
Content-Length: 11
254+
Content-Type: text/plain
255+
```
256+
257+
### Step 3 — Add a Dockerfile
258+
259+
Create `src/Servers/YourServer/Dockerfile` that builds and runs the server.
260+
261+
Key requirements:
262+
- The container runs with `--network host`, so bind to `0.0.0.0:8080`.
263+
- Use `ENTRYPOINT` (not `CMD`) for the server process.
264+
- The Dockerfile build context is the repository root, so paths like `COPY src/Servers/YourServer/...` are correct.
265+
266+
Example:
267+
```dockerfile
268+
FROM python:3.12-slim
269+
WORKDIR /app
270+
RUN pip install --no-cache-dir flask
271+
COPY src/Servers/YourServer/app.py .
272+
ENTRYPOINT ["python3", "app.py", "8080"]
273+
```
274+
275+
### Step 4 — Add probe.json
276+
277+
Create `src/Servers/YourServer/probe.json` with exactly one field:
278+
279+
```json
280+
{"name": "Your Server Display Name"}
281+
```
282+
283+
This name appears in the leaderboard and PR comments.
284+
285+
### Verification checklist
286+
287+
1. Build the Docker image: `docker build -f src/Servers/YourServer/Dockerfile -t yourserver .`
288+
2. Run: `docker run --network host yourserver`
289+
3. Verify endpoints:
290+
- `curl http://localhost:8080/` returns 200
291+
- `curl -X POST -d "hello" http://localhost:8080/` returns "hello"
292+
- `curl -X POST -d "test" http://localhost:8080/echo` returns headers
293+
4. Run the probe: `dotnet run --project src/Http11Probe.Cli -- --host localhost --port 8080`
294+
295+
No changes to CI workflows, configs, or other files are needed. The pipeline auto-discovers servers from `src/Servers/*/probe.json`.

‎CHANGELOG.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,15 @@
22

33
All notable changes to Http11Probe are documented in this file.
44

5-
## [Unreleased]
5+
## [2026-02-14]
66

77
### Added
8+
- **RFC Requirement Dashboard** — all 148 tests classified by RFC 2119 level (MUST/SHOULD/MAY/"ought to"/Unscored/N/A) with exact RFC quotes proving each classification (`docs/content/docs/rfc-requirement-dashboard.md`)
9+
- **Add a Test guide** — step-by-step documentation for contributing new tests to the platform (`docs/content/add-a-test.md`)
10+
- **AI Agent contribution guide** — machine-readable `AGENTS.md` at repo root with precise instructions for LLM agents to add tests or frameworks (`docs/content/add-with-ai-agent.md`)
11+
- **Contribute menu** — top nav "Add a Framework" replaced with a "Contribute" dropdown containing Add a Framework, Add a Test, and Add with AI Agent
12+
- **Landing page cards** — RFC Requirement Dashboard card in hero section; Add a Test and Add with AI Agent cards in Contribute section
13+
- **Glossary card** — RFC Requirement Dashboard linked from the glossary index page
814
- **Server configuration pages** — per-server docs pages showing Dockerfile, source code, and config files for all 36 tested servers (`docs/content/servers/`)
915
- **Clickable server names** — server names in the probe results table and summary bar chart now link to their configuration page
1016
- **Sticky first column** — server name column stays pinned to the left edge while scrolling horizontally through result tables
@@ -33,6 +39,9 @@ All notable changes to Http11Probe are documented in this file.
3339
- **Stronger sticky shadow on mobile** — increased shadow intensity for the pinned server name column
3440
- **Scrollable tooltips** — hover tooltips are now interactive and scrollable for large payloads (removed `pointer-events:none`, increased `max-height` to `60vh`)
3541
- **Larger click modal** — expanded from `max-width:700px` to `90vw` and `max-height` from `80vh` to `85vh` to better accommodate large request/response data
42+
- **Landing page section rename** — "Add Your Framework" heading renamed to "Contribute to the Project" with updated copy emphasizing community contributions
43+
- **Uniform card sizing** — CSS rule forces all home page card grids to `repeat(2, 1fr)` so every card is the same width
44+
- **Sidebar reordering** — RFC Requirement Dashboard at weight 2 (after Understanding HTTP), RFC Basics bumped to 3, Baseline to 4
3645
- **Kestrel HEAD/OPTIONS support** — added explicit HEAD and OPTIONS endpoint handlers to ASP.NET Minimal server so smuggling tests evaluate correctly instead of returning 405
3746
- **Add a Framework docs** — documented HEAD and OPTIONS as required endpoints
3847
- Raw request capture now includes truncation metadata when payload exceeds 8,192 bytes (`TestRunner.cs`)

‎docs/assets/css/custom.css‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,12 @@ html.dark .hextra-content {
4545
background: transparent !important;
4646
}
4747

48+
/* ── Home page: uniform card grid ────────────────────────────── */
49+
50+
.hextra-home-body .hextra-cards {
51+
grid-template-columns: repeat(2, 1fr) !important;
52+
}
53+
4854
/* ── Cards ───────────────────────────────────────────────────── */
4955

5056
.hextra-card {

‎docs/content/_index.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ layout: hextra-home
2424

2525
{{< cards >}}
2626
{{< card link="probe-results" title="Leaderboard" subtitle="See which frameworks pass the most tests, ranked from best to worst compliance." icon="chart-bar" >}}
27+
{{< card link="docs/rfc-requirement-dashboard" title="RFC Requirement Dashboard" subtitle="All 148 tests classified by RFC 2119 level (MUST/SHOULD/MAY)." icon="document-search" >}}
2728
{{< /cards >}}
2829

2930
<div style="height:60px"></div>
@@ -45,15 +46,19 @@ Http11Probe sends a suite of crafted HTTP requests to each server and checks whe
4546

4647
<div style="height:60px"></div>
4748

48-
<h2 style="font-size:2rem;font-weight:800;">Add Your Framework</h2>
49+
<h2 style="font-size:2rem;font-weight:800;">Contribute to the Project</h2>
4950

5051
<div style="height:16px"></div>
5152

52-
Http11Probe is designed so anyone can contribute their HTTP server and get compliance results without touching the test infrastructure. Just add a Dockerfile, a one-line `probe.json`, and open a PR.
53+
Http11Probe is open source and built for contributions. Add your HTTP server to the leaderboard, or write new test cases to expand coverage.
54+
55+
Every new framework added makes the comparison more useful for the entire community, and every new test strengthens the compliance bar for all servers on the platform. If you've found an edge case that isn't covered, or you maintain a framework that isn't listed yet, your contribution directly improves HTTP security and interoperability for everyone.
5356

5457
<div style="height:20px"></div>
5558

5659
{{< cards >}}
57-
{{< card link="add-a-framework" title="Get Started" subtitle="Three steps to add your framework — Dockerfile, probe.json, and open a PR." icon="plus-circle" >}}
60+
{{< card link="add-a-framework" title="Add a Framework" subtitle="Three steps to add your framework — Dockerfile, probe.json, and open a PR." icon="plus-circle" >}}
61+
{{< card link="add-a-test" title="Add a Test" subtitle="How to define a new test case, write its documentation, and wire it into the platform." icon="beaker" >}}
62+
{{< card link="add-with-ai-agent" title="Add with AI Agent" subtitle="Use an AI coding agent to add a test or framework using the machine-readable AGENTS.md guide." icon="chip" >}}
5863
{{< /cards >}}
5964

0 commit comments

Comments
 (0)