fix: add explicit model for Copilot BYOK smoke test (#2006)

* fix: add explicit model for Copilot BYOK mode

Copilot CLI requires --model when running in BYOK/offline mode.
Without it, all attempts fail with: 'BYOK providers require an
explicit model.'

Set COPILOT_MODEL=gpt-4.1 in workflow env and override the
compiler's empty default fallback in the lock file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: inject Copilot-Integration-Id header in api-proxy

api.githubcopilot.com requires the Copilot-Integration-Id header.
In BYOK mode, the Copilot CLI sends plain OpenAI-compatible
requests without Copilot-specific headers. The api-proxy sidecar
must inject this header when proxying to the Copilot API.

Uses 'copilot-developer-cli' as the default integration ID,
matching the Copilot CLI's own value. Configurable via
COPILOT_INTEGRATION_ID env var.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: align BYOK smoke test with copilot smoke test workload

- Add GitHub.com connectivity check and pre-fetched PR data comparison
- Add all 4 test sections: MCP, connectivity, file I/O, BYOK inference
- Update model from gpt-4.1 to gpt-5.4
- Recompile and post-process lock file

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: switch BYOK model to gpt-5.3-codex

gpt-5.4 returns 400 on tools[5].function.name empty string,
likely a stricter validation in that model. Try gpt-5.3-codex.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: recompile lock file for gpt-5.3-codex model change

Previous commit only patched the lock file without recompiling,
causing frontmatter hash mismatch at runtime.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: switch BYOK model to claude-opus-4.6

gpt-5.3-codex is not accessible via /chat/completions endpoint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump Copilot CLI to 1.0.28 for BYOK testing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: lpcox

.github/workflows/smoke-copilot-byok.lock.yml

@@ -38,6 +38,7 @@ safe-outputs:
 timeout-minutes: 5
 env:
   COPILOT_API_KEY: dummy-byok-key-for-offline-mode
+  COPILOT_MODEL: claude-opus-4.6
 features:
   cli-proxy: true
 strict: true
@@ -50,6 +51,18 @@ steps:
       echo "COPILOT_API_TARGET=${COPILOT_API_TARGET:-api.githubcopilot.com (default)}"
       echo "::endgroup::"
 
+      echo "::group::Fetching last 2 merged PRs"
+      PR_DATA=$(gh pr list --repo "$GITHUB_REPOSITORY" --state merged --limit 2 \
+        --json number,title,author,mergedAt \
+        --jq '.[] | "PR #\(.number): \(.title) (by @\(.author.login), merged \(.mergedAt))"')
+      echo "$PR_DATA"
+      echo "::endgroup::"
+
+      echo "::group::GitHub.com connectivity check"
+      HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 10 https://github.com)
+      echo "github.com returned HTTP $HTTP_CODE"
+      echo "::endgroup::"
+
       echo "::group::File write/read test"
       TEST_DIR="/tmp/gh-aw/agent"
       TEST_FILE="$TEST_DIR/smoke-test-copilot-byok-${GITHUB_RUN_ID}.txt"
@@ -60,9 +73,15 @@ steps:
       echo "::endgroup::"
 
       {
+        echo "SMOKE_PR_DATA<<SMOKE_EOF"
+        echo "$PR_DATA"
+        echo "SMOKE_EOF"
+        echo "SMOKE_HTTP_CODE=$HTTP_CODE"
         echo "SMOKE_FILE_CONTENT=$FILE_CONTENT"
         echo "SMOKE_FILE_PATH=$TEST_FILE"
       } >> "$GITHUB_OUTPUT"
+    env:
+      GH_TOKEN: ${{ github.token }}
 post-steps:
   - name: Validate safe outputs were invoked
     run: |
@@ -105,21 +124,34 @@ This smoke test validates that Copilot CLI runs in **offline BYOK mode** — wit
 
 ## Pre-Computed Test Results
 
-### 1. File Write/Read Test
+The following tests were already executed in a deterministic pre-agent step. Your job is to verify the results and produce the summary comment.
+
+### 1. GitHub MCP Testing
+Verify MCP connectivity by calling `github-list_pull_requests` for ${{ github.repository }} (limit 1, state merged). Confirm the result matches the pre-fetched data below.
+
+### 2. GitHub.com Connectivity
+Pre-step result: HTTP ${{ steps.smoke-data.outputs.SMOKE_HTTP_CODE }} from github.com.
+✅ if HTTP 200 or 301, ❌ otherwise.
+
+### 3. File Write/Read Test
 Pre-step wrote and read back: "${{ steps.smoke-data.outputs.SMOKE_FILE_CONTENT }}"
 File path: ${{ steps.smoke-data.outputs.SMOKE_FILE_PATH }}
 Verify by running `cat` on the file path using bash to confirm it exists.
 
-### 2. GitHub MCP Testing
-Verify MCP connectivity by calling `github-list_pull_requests` for ${{ github.repository }} (limit 1, state merged). Confirm the result returns data.
-
-### 3. BYOK Inference Test
+### 4. BYOK Inference Test
 You are running in offline BYOK mode right now. The fact that you can read this prompt and respond means the BYOK inference path (agent → api-proxy sidecar → api.githubcopilot.com) is working. Confirm ✅.
 
+## Pre-Fetched PR Data
+
+```
+${{ steps.smoke-data.outputs.SMOKE_PR_DATA }}
+```
+
 ## Output
 
 Add a **very brief** comment (max 5-10 lines) to the current pull request with:
-- ✅ or ❌ for each test result (file I/O, MCP, inference)
+- PR titles only (no descriptions)
+- ✅ or ❌ for each test result
 - Note: "Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com"
 - Overall status: PASS or FAIL
 - Mention the pull request author and any assignees

.github/workflows/smoke-copilot-byok.md

@@ -78,6 +78,7 @@ function resolveCopilotAuthToken(env = process.env) {
 }
 
 const COPILOT_AUTH_TOKEN = resolveCopilotAuthToken(process.env);
+const COPILOT_INTEGRATION_ID = process.env.COPILOT_INTEGRATION_ID || 'copilot-developer-cli';
 const GEMINI_API_KEY = (process.env.GEMINI_API_KEY || '').trim() || undefined;
 
 /**
@@ -337,7 +338,7 @@ function resolveOpenCodeRoute(openaiKey, anthropicKey, copilotToken, openaiTarge
     return { target: anthropicTarget, headers: { 'x-api-key': anthropicKey }, basePath: anthropicBasePath, needsAnthropicVersion: true };
   }
   if (copilotToken) {
-    return { target: copilotTarget, headers: { 'Authorization': `Bearer ${copilotToken}` }, basePath: undefined, needsAnthropicVersion: false };
+    return { target: copilotTarget, headers: { 'Authorization': `Bearer ${copilotToken}`, 'Copilot-Integration-Id': COPILOT_INTEGRATION_ID }, basePath: undefined, needsAnthropicVersion: false };
   }
   return null;
 }
@@ -1012,18 +1013,21 @@ if (require.main === module) {
       if (isModelsPath && req.method === 'GET' && COPILOT_GITHUB_TOKEN) {
         proxyRequest(req, res, COPILOT_API_TARGET, {
           'Authorization': `Bearer ${COPILOT_GITHUB_TOKEN}`,
+          'Copilot-Integration-Id': COPILOT_INTEGRATION_ID,
         }, 'copilot');
         return;
       }
 
       proxyRequest(req, res, COPILOT_API_TARGET, {
         'Authorization': `Bearer ${COPILOT_AUTH_TOKEN}`,
+        'Copilot-Integration-Id': COPILOT_INTEGRATION_ID,
       }, 'copilot');
     });
 
     copilotServer.on('upgrade', (req, socket, head) => {
       proxyWebSocket(req, socket, head, COPILOT_API_TARGET, {
         'Authorization': `Bearer ${COPILOT_AUTH_TOKEN}`,
+        'Copilot-Integration-Id': COPILOT_INTEGRATION_ID,
       }, 'copilot');
     });