Skip to content

fix: re-add --no-seccomp on ARM64 for UFFD snapshot restore#2391

Closed
tomassrnka wants to merge 1 commit intomainfrom
fix/arm64-no-seccomp
Closed

fix: re-add --no-seccomp on ARM64 for UFFD snapshot restore#2391
tomassrnka wants to merge 1 commit intomainfrom
fix/arm64-no-seccomp

Conversation

@tomassrnka
Copy link
Copy Markdown
Member

Summary

Re-adds --no-seccomp for Firecracker on ARM64. The aarch64 seccomp filter is missing a syscall needed during UFFD-based snapshot restore, causing FC to exit with code 148 (BadSyscall).

Investigation findings

Scenario Seccomp Result
Basic boot + file-based restore Enabled Pass (exit 0)
UFFD-based restore (orchestrator) Enabled Fail (exit 148)
UFFD-based restore SECCOMP_RET_LOG filter Pass (build proceeds)
UFFD-based restore --no-seccomp Pass
  • FC exit code 148 = FcExitCode::BadSyscall (Firecracker's seccomp trap handler)
  • The blocked syscall only triggers during UFFD-based snapshot restore, not file-based
  • Switching the seccomp default action from trap to log allows the build to succeed, confirming seccomp is the blocker
  • The exact blocked syscall number is TBD (strace too slow with UFFD, no perf tools for this kernel)

Reproduction

On an ARM64 VM with KVM (e.g. Lima VZ on Apple Silicon M3+):

# Template build fails with default seccomp:
make -C packages/shared/scripts local-build-base-template
# → orchestrator log shows: "error waiting for fc process: exit status 148"

# Works with --no-seccomp or SECCOMP_RET_LOG custom filter

Upstream

To be filed against firecracker-microvm/firecracker — aarch64 seccomp filter missing syscall for UFFD snapshot restore path.

Test plan

  • ARM64 template build succeeds with --no-seccomp
  • x86_64 builds unaffected (no --no-seccomp added)

Firecracker's aarch64 seccomp filter is missing a syscall needed
during UFFD-based snapshot restore. FC exits with code 148
(BadSyscall) when restoring snapshots via the orchestrator's UFFD
handler on ARM64.

Findings:
- Basic boot + file-based restore: works with seccomp
- UFFD-based restore (used by orchestrator): fails with exit 148
- Using SECCOMP_RET_LOG filter (log instead of trap): build succeeds
- Confirms a specific syscall in the UFFD path is blocked

This is an upstream Firecracker issue — the aarch64 seccomp filter
needs to be updated. --no-seccomp is the workaround until then.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 14, 2026

PR Summary

Medium Risk
Medium risk because it conditionally disables a key sandboxing layer (seccomp) for all ARM64 Firecracker runs, expanding the attack surface if untrusted workloads can reach the microVM boundary.

Overview
This changes the generated Firecracker start script to append --no-seccomp on arm64 by adding an ExtraArgs field to the script template args and populating it based on runtime.GOARCH, restoring UFFD-based snapshot restore behavior on ARM64.

Reviewed by Cursor Bugbot for commit 895b07a. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review comment

@tomassrnka tomassrnka closed this Apr 14, 2026
// Tracked upstream: https://github.com/firecracker-microvm/firecracker/issues/XXXX
// TODO: remove once upstream adds the missing syscall to the aarch64 filter.
var extraArgs string
if runtime.GOARCH == "arm64" {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code comment notes that basic boot and file-based restore work fine with seccomp enabled -- only the UFFD restore path triggers the violation. But --no-seccomp is unconditionally applied to all Firecracker invocations on ARM64 (fresh boots, file-based restores, and UFFD restores alike), disabling syscall filtering in cases where it would have worked correctly.

If UFFD restore invocations can be distinguished from other invocations (e.g. a flag passed through buildArgs), limiting --no-seccomp to just the UFFD restore path would reduce the security regression surface. If the code path is genuinely shared and cannot be split, that is worth documenting explicitly here.

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This disables seccomp for all ARM64 Firecracker runs as a workaround for a missing syscall in the aarch64 filter — worth a human look given the security trade-off, even though the fix itself is well-investigated and cleanly implemented.

Extended reasoning...

Overview

Single-file change to script_builder.go that adds an ExtraArgs field to the start script template and populates it with --no-seccomp when runtime.GOARCH == "arm64". Both V1 and V2 script templates are updated consistently.

Security Risks

The change explicitly disables Firecracker's seccomp filter on all ARM64 builds, not just the UFFD restore path. Seccomp is a meaningful defense-in-depth layer inside the microVM boundary. While the PR author has done thorough investigation (exit code 148, SECCOMP_RET_LOG test, --no-seccomp confirmation), the exact blocked syscall is still TBD and no upstream issue has been filed yet (placeholder XXXX). Disabling seccomp for all ARM64 deployments warrants explicit sign-off from someone who owns the security posture of the sandbox.

Level of Scrutiny

Medium-high. The change is small and mechanical but carries a meaningful security implication. The investigation is solid and the workaround is a known pattern, but this re-adds a broad bypass rather than a narrow fix. Human confirmation of the trade-off is appropriate here.

Other Factors

No bugs were found. The code is well-commented with reason, upstream tracking intent, and a TODO to remove when upstream fixes the filter. The title says "re-add" suggesting this was present before, which somewhat lowers the risk.

@ValentaTomas ValentaTomas deleted the fix/arm64-no-seccomp branch April 18, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants