Optimize Smoke Claude workflow token spend (Haiku model, tighter turn cap, narrower GitHub toolset) by Copilot · Pull Request #2065 · github/gh-aw-firewall

Copilot · 2026-04-18T01:08:47Z

The Smoke Claude workflow was over-provisioned for its task profile, with cost dominated by large tool schema/cache writes and a higher-capability model than required. This PR reduces per-run token cost by switching to a lighter model and tightening execution/tool scope.

Engine cost controls
- Set Claude model to claude-haiku-4-5
- Reduced max-turns from 12 to 8
GitHub MCP scope reduction
- Narrowed GitHub toolsets from [repos, pull_requests] to [pull_requests] to reduce loaded tool schema surface
Workflow source updated
- Changes are applied in:
  - .github/workflows/smoke-claude.md

engine:
  id: claude
  model: claude-haiku-4-5
  max-turns: 8

tools:
  github:
    toolsets: [pull_requests]

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/graphql
- Triggering command: /usr/bin/gh gh repo view github/gh-aw --json name,defaultBranchRef (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.68.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/12320643-5e08-4b46-84d1-2c6428335c6f

Copilot

Pull request overview

Reduces the Smoke Claude agentic workflow’s token spend by switching to a lighter Claude model, lowering the turn cap, and narrowing the GitHub MCP tool surface.

Changes:

Set Claude engine model to claude-haiku-4-5 and reduced max-turns from 12 to 8.
Narrowed GitHub MCP toolsets from [repos, pull_requests] to [pull_requests].
Minor formatting tweak in the workflow’s markdown instructions.

Show a summary per file

File	Description
.github/workflows/smoke-claude.md	Adjusts Claude engine config and GitHub toolset scope to reduce per-run token usage.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 1/1 changed files
Comments generated: 1

Copilot · 2026-04-18T01:18:54Z

 engine:
  id: claude
-  max-turns: 12
+  model: claude-haiku-4-5
+  max-turns: 8


These updates are in the .md source, but the workflow that actually runs is .github/workflows/smoke-claude.lock.yml (which is currently still generated with --max-turns 12 and the old GitHub tool allowlist). Please re-run gh aw compile and commit the regenerated smoke-claude.lock.yml so the model/max-turns/toolset changes take effect and the lock freshness check doesn’t fail.

lpcox · 2026-04-18T01:19:55Z

@copilot address this review feedback #2065 (review)

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/74ba0807-1761-43a1-bfb1-ad7a853aa1bf

Copilot · 2026-04-18T01:24:58Z

@copilot address this review feedback #2065 (review)

Addressed in commit 3881ad0: updated .github/workflows/smoke-claude.lock.yml so the runtime now reflects the optimization changes (Haiku model, max-turns 8, and narrowed GitHub MCP allowed tools for PR listing/search).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/repos/actions/github-script/git/ref/tags/v9
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.68.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha (http block)
https://api.github.com/repos/github/gh-aw-actions/releases/tags/v0.68.7
- Triggering command: /usr/bin/gh gh api repos/github/gh-aw-actions/releases/tags/v0.68.7 (http block)
https://api.github.com/repos/github/gh-aw/releases/tags/v0.68.7
- Triggering command: /usr/bin/gh gh api repos/github/gh-aw/releases/tags/v0.68.7 (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

github-actions · 2026-04-18T01:50:47Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	84.33%	84.41%	📈 +0.08%
Statements	83.56%	83.64%	📈 +0.08%
Functions	87.39%	87.39%	➡️ +0.00%
Branches	74.78%	74.82%	📈 +0.04%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	86.8% → 87.1% (+0.30%)	86.4% → 86.7% (+0.29%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-04-18T01:52:33Z

Smoke Test: Copilot BYOK (Offline) Mode

Test	Result
GitHub MCP (`list_pull_requests` → PR #2062 "Copilot/convert cli proxy to byok")	✅
GitHub.com connectivity	✅
File write/read	✅
BYOK inference (agent → api-proxy → api.githubcopilot.com)	✅

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: PASS

CC @lpcox @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-04-18T01:52:47Z

🔥 Smoke Test Results

MCP (github-list_pull_requests): ✅ Last merged PR: Copilot/convert cli proxy to byok #2062 "Copilot/convert cli proxy to byok"
GitHub.com connectivity: ✅ (file written/read confirms network access)
File write/read: ✅ smoke-test-copilot-24593695588.txt verified

Overall: PASS

Author: @app/copilot-swe-agent | Assignees: @lpcox, @Copilot

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-04-18T01:52:50Z

Smoke Test Results

✅ GitHub MCP: Last 2 merged PRs retrieved

Copilot/convert cli proxy to byok #2062: Copilot/convert cli proxy to byok
feat: convert remaining cli-proxy workflows to byok-copilot #2058: feat: convert remaining cli-proxy workflows to byok-copilot

✅ Playwright: github.com page title verified
✅ File Writing: Test file created at /tmp/gh-aw/agent/
✅ Bash Verification: File contents confirmed

Status: PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-04-18T01:54:04Z

Smoke Test: GitHub Actions Services Connectivity

Check	Status	Details
Redis PING (`host.docker.internal:6379`)	✅ Pass	`+PONG`
`pg_isready` (`host.docker.internal:5432`)	✅ Pass	accepting connections
`psql SELECT 1` (`smoketest` db)	✅ Pass	returned `1`

All checks passed. (redis-cli unavailable; Redis verified via raw socket.)

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-04-18T01:55:36Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #2065 · ● 594.1K · ◷

Initial plan

26c9dae

Copilot AI assigned Copilot and lpcox Apr 18, 2026

Copilot started work on behalf of lpcox April 18, 2026 01:08 View session

Copilot AI linked an issue Apr 18, 2026 that may be closed by this pull request

⚡ Claude Token Optimization2026-04-17 — Smoke Claude #2055

Closed

9 tasks

Copilot AI added 2 commits April 18, 2026 01:12

ci: optimize smoke-claude model turns and github toolset

6da384b

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/12320643-5e08-4b46-84d1-2c6428335c6f

chore: revert unintended actions-lock changes

3f2197c

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/12320643-5e08-4b46-84d1-2c6428335c6f

Copilot AI changed the title ~~[WIP] Optimize token usage in smoke Claude workflow~~ Optimize Smoke Claude workflow token spend (Haiku model, tighter turn cap, narrower GitHub toolset) Apr 18, 2026

Copilot AI requested a review from lpcox April 18, 2026 01:14

Copilot finished work on behalf of lpcox April 18, 2026 01:14

lpcox marked this pull request as ready for review April 18, 2026 01:15

lpcox requested a review from Mossaka as a code owner April 18, 2026 01:15

Copilot AI review requested due to automatic review settings April 18, 2026 01:15

github-actions bot mentioned this pull request Apr 18, 2026

[aw] Smoke Claude failed #2067

Closed

Copilot started reviewing on behalf of lpcox April 18, 2026 01:15 View session

github-actions bot added the smoke-copilot-byok label Apr 18, 2026

This comment has been minimized.

Sign in to view

github-actions bot added the smoke-copilot label Apr 18, 2026

This comment has been minimized.

Sign in to view

Copilot AI reviewed Apr 18, 2026

View reviewed changes

github-actions bot mentioned this pull request Apr 18, 2026

[aw] No-Op Runs #2069

Closed

This comment has been minimized.

Sign in to view

github-actions bot added the smoke-services label Apr 18, 2026

github-actions bot mentioned this pull request Apr 18, 2026

[aw] Smoke Codex failed #2070

Closed

Copilot started work on behalf of lpcox April 18, 2026 01:20 View session

This comment has been minimized.

Sign in to view

github-actions bot added the build-test label Apr 18, 2026

ci: update smoke-claude lock with model turns and trimmed github tools

3881ad0

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/74ba0807-1761-43a1-bfb1-ad7a853aa1bf

Copilot finished work on behalf of lpcox April 18, 2026 01:25

github-actions bot added the smoke-claude label Apr 18, 2026

lpcox merged commit 49e9508 into main Apr 18, 2026
52 of 56 checks passed

lpcox deleted the copilot/optimize-claude-token-usage-again branch April 18, 2026 01:54

This was referenced Apr 18, 2026

[awf] API proxy: correct OpenAI base URL injection for Codex /v1/responses routing #2066

Merged

[Deps] Safe patch/minor devDependency refresh (2026-04-17) + Node minimum alignment #2075

Merged

Conversation

Copilot AI commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

lpcox commented Apr 18, 2026

Uh oh!

This comment has been minimized.

Copilot AI commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions bot commented Apr 18, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions bot commented Apr 18, 2026

Smoke Test: Copilot BYOK (Offline) Mode

Uh oh!

github-actions bot commented Apr 18, 2026

🔥 Smoke Test Results

Uh oh!

github-actions bot commented Apr 18, 2026

Uh oh!

github-actions bot commented Apr 18, 2026

Smoke Test: GitHub Actions Services Connectivity

Uh oh!

Uh oh!

github-actions bot commented Apr 18, 2026

🏗️ Build Test Suite Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 18, 2026 •

edited

Loading

Copilot AI commented Apr 18, 2026 •

edited

Loading