Skip to content

Commit 410a3e6

Browse files
daandemeyerdaandemeyer
committed
Add workflow to automatically update distribution snapshots
Add a GitHub Actions workflow that runs every three days to pin each distribution to its latest snapshot. To make this work, we extend the latest-snapshot verb with the option to write the snapshot to a config file provided by the user. We do this to avoid having to start tracking which config file provided a setting. Even if we had that, we still would need the explicit config file to bootstrap the initial write so we might as well insist on the user providing it. The workflow runs mkosi latest-snapshot for each distribution (arch, centos, debian, fedora, opensuse, ubuntu) which fetches the latest snapshot, updates the corresponding mkosi config file with the new Snapshot= setting, and commits the result. The workflow then creates a pull request for the update using github-script. Each distribution gets its own dedicated branch (update-snapshot/<distro>) so that only a single pull request is open per distribution at a time. If an existing pull request is already open, it is updated in place by force-pushing to the branch. Auto-merge is enabled on each pull request via the GraphQL enablePullRequestAutoMerge mutation so that the snapshot update lands automatically once CI is green.
1 parent 9bbeb52 commit 410a3e6

6 files changed

Lines changed: 407 additions & 4 deletions

File tree

‎.github/actions/mkosi-update-snapshot/action.yml‎

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
# SPDX-License-Identifier: LGPL-2.1-or-later
2+
name: mkosi-update-snapshot
3+
description: Update distribution snapshot and create a pull request
4+
5+
inputs:
6+
token:
7+
description: GitHub token for authentication
8+
required: true
9+
distribution:
10+
description: Distribution to update the snapshot for
11+
required: false
12+
default: ""
13+
config:
14+
description: Path to the config file to update
15+
required: true
16+
release:
17+
description: Distribution release to update the snapshot for
18+
required: false
19+
default: ""
20+
21+
runs:
22+
using: composite
23+
steps:
24+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
25+
with:
26+
token: ${{ inputs.token }}
27+
28+
- uses: ./
29+
30+
- name: Free disk space
31+
shell: bash
32+
run: |
33+
sudo mv /usr/local /usr/local.trash
34+
sudo mv /opt/hostedtoolcache /opt/hostedtoolcache.trash
35+
sudo systemd-run rm -rf /usr/local.trash /opt/hostedtoolcache.trash
36+
37+
- name: Configure git
38+
shell: bash
39+
run: |
40+
git config user.name "github-actions[bot]"
41+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
42+
43+
- name: Update snapshot
44+
id: update
45+
shell: bash
46+
env:
47+
DISTRIBUTION: ${{ inputs.distribution }}
48+
RELEASE: ${{ inputs.release }}
49+
run: |
50+
branch="update-snapshot/${DISTRIBUTION:-default}"
51+
git checkout -B "$branch"
52+
53+
mkosi ${DISTRIBUTION:+-d "$DISTRIBUTION"} ${RELEASE:+-r "$RELEASE"} latest-snapshot -- --update ${{ inputs.config }} --commit
54+
55+
if git diff --quiet HEAD~1 HEAD 2>/dev/null; then
56+
echo "updated=false" >>"$GITHUB_OUTPUT"
57+
else
58+
echo "updated=true" >>"$GITHUB_OUTPUT"
59+
echo "branch=$branch" >>"$GITHUB_OUTPUT"
60+
echo "title=$(git log -1 --format=%s)" >>"$GITHUB_OUTPUT"
61+
fi
62+
63+
- name: Build image and generate manifest diff
64+
if: steps.update.outputs.updated == 'true'
65+
shell: bash
66+
env:
67+
DISTRIBUTION: ${{ inputs.distribution }}
68+
RELEASE: ${{ inputs.release }}
69+
run: |
70+
tee mkosi.local.conf <<EOF
71+
[Distribution]
72+
${DISTRIBUTION:+Distribution=$DISTRIBUTION}
73+
${RELEASE:+Release=$RELEASE}
74+
75+
[Output]
76+
Format=directory
77+
ManifestFormat=json
78+
EOF
79+
80+
# Build the image to generate a manifest. If the build fails, we still want to
81+
# create the PR with just the snapshot update but include the build output so
82+
# that it's easy to see what went wrong.
83+
if ! mkosi -f 2>&1 | tee /tmp/mkosi-build.log; then
84+
echo "::warning::Image build failed, skipping manifest diff"
85+
cat >/tmp/pr-body.md <<BODY
86+
:x: **Image build failed**
87+
88+
<details>
89+
<summary>Build log</summary>
90+
91+
\`\`\`
92+
$(cat /tmp/mkosi-build.log)
93+
\`\`\`
94+
95+
</details>
96+
BODY
97+
mkosi -f clean
98+
git push -f origin "${{ steps.update.outputs.branch }}"
99+
exit 0
100+
fi
101+
102+
manifest="mkosi.output/image.manifest"
103+
104+
if [[ -e mkosi/mkosi.conf ]] || [[ -e mkosi/mkosi.tools.conf ]]; then
105+
manifests_dir="mkosi/manifests"
106+
else
107+
manifests_dir="manifests"
108+
fi
109+
110+
old_manifest="$manifests_dir/${DISTRIBUTION:-default}.manifest.json"
111+
112+
if [[ -f "$old_manifest" && -f "$manifest" ]]; then
113+
python3 tools/mkosi-manifest-diff.py "$old_manifest" "$manifest" >/tmp/pr-body.md
114+
elif [[ -f "$manifest" ]]; then
115+
echo "First manifest for this distribution, no previous manifest to diff against." >/tmp/pr-body.md
116+
fi
117+
118+
if [[ -f "$manifest" ]]; then
119+
mkdir -p "$manifests_dir"
120+
cp "$manifest" "$old_manifest"
121+
git add "$old_manifest"
122+
git commit --amend --no-edit
123+
fi
124+
125+
mkosi -f clean
126+
rm mkosi.local.conf
127+
git push -f origin "${{ steps.update.outputs.branch }}"
128+
129+
- name: Create or update pull request
130+
if: steps.update.outputs.updated == 'true'
131+
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea
132+
with:
133+
github-token: ${{ inputs.token }}
134+
script: |
135+
const fs = require('fs');
136+
const branch = '${{ steps.update.outputs.branch }}';
137+
const title = '${{ steps.update.outputs.title }}';
138+
139+
let body = '';
140+
try {
141+
body = fs.readFileSync('/tmp/pr-body.md', 'utf8').trim();
142+
} catch (e) {
143+
console.log('No PR body file found, continuing without body');
144+
}
145+
146+
const { data: pulls } = await github.rest.pulls.list({
147+
owner: context.repo.owner,
148+
repo: context.repo.repo,
149+
state: 'open',
150+
head: `${context.repo.owner}:${branch}`,
151+
});
152+
153+
let number;
154+
155+
if (pulls.length > 0) {
156+
const pull = pulls[0];
157+
await github.rest.pulls.update({
158+
owner: context.repo.owner,
159+
repo: context.repo.repo,
160+
pull_number: pull.number,
161+
title: title,
162+
body: body,
163+
});
164+
console.log(`Updated existing pull request #${pull.number}`);
165+
number = pull.number;
166+
} else {
167+
const { data: pr } = await github.rest.pulls.create({
168+
owner: context.repo.owner,
169+
repo: context.repo.repo,
170+
title: title,
171+
head: branch,
172+
base: 'main',
173+
body: body,
174+
});
175+
console.log(`Created pull request #${pr.number}`);
176+
number = pr.number;
177+
}
178+
179+
// Enable auto-merge so the PR merges automatically once CI is green.
180+
try {
181+
const { data: pullData } = await github.rest.pulls.get({
182+
owner: context.repo.owner,
183+
repo: context.repo.repo,
184+
pull_number: number,
185+
});
186+
187+
await github.graphql(`
188+
mutation($pullRequestId: ID!) {
189+
enablePullRequestAutoMerge(input: { pullRequestId: $pullRequestId, mergeMethod: MERGE }) {
190+
clientMutationId
191+
}
192+
}
193+
`, { pullRequestId: pullData.node_id });
194+
195+
console.log(`Enabled auto-merge for pull request #${number}`);
196+
} catch (e) {
197+
console.log(`Could not enable auto-merge for pull request #${number}: ${e.message}`);
198+
}

‎.github/workflows/update-snapshot.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# SPDX-License-Identifier: LGPL-2.1-or-later
2+
name: update-snapshot
3+
4+
on:
5+
schedule:
6+
# Update snapshots every three days at midnight
7+
- cron: "0 0 */3 * *"
8+
workflow_dispatch:
9+
10+
jobs:
11+
update-snapshot:
12+
runs-on: ubuntu-24.04
13+
strategy:
14+
fail-fast: false
15+
matrix:
16+
include:
17+
- distro: arch
18+
config: mkosi.conf.d/arch.conf
19+
- distro: centos
20+
config: mkosi.conf.d/centos.conf
21+
- distro: debian
22+
config: mkosi.conf.d/debian.conf
23+
- distro: fedora
24+
config: mkosi.conf.d/fedora/mkosi.conf
25+
- distro: opensuse
26+
config: mkosi.conf.d/opensuse/mkosi.conf
27+
# Ubuntu does not have snapshots for the "devel" release
28+
# - distro: ubuntu
29+
# config: mkosi.conf.d/ubuntu.conf
30+
31+
steps:
32+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
33+
with:
34+
persist-credentials: false
35+
36+
- uses: actions/create-github-app-token@v2
37+
id: token
38+
with:
39+
app-id: ${{ vars.TRIGGER_CHECKS_ON_WORKFLOW_APP_ID }}
40+
private-key: ${{ secrets.TRIGGER_CHECKS_ON_WORKFLOW_PRIVATE_KEY }}
41+
42+
- uses: ./.github/actions/mkosi-update-snapshot
43+
with:
44+
token: ${{ steps.token.outputs.token }}
45+
distribution: ${{ matrix.distro }}
46+
config: ${{ matrix.config }}

‎mkosi/__init__.py‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@
139139
unshare,
140140
userns_has_single_user,
141141
)
142+
from mkosi.snapshot import run_latest_snapshot
142143
from mkosi.sysupdate import run_sysupdate
143144
from mkosi.tree import copy_tree, is_foreign_uid_tree, make_tree, move_tree, rmtree
144145
from mkosi.user import INVOKING_USER, become_root_cmd
@@ -4271,10 +4272,6 @@ def run_box(args: Args, config: Config) -> None:
42714272
)
42724273

42734274

4274-
def run_latest_snapshot(args: Args, config: Config) -> None:
4275-
print(config.distribution.installer.latest_snapshot(config))
4276-
4277-
42784275
def run_shell(args: Args, config: Config) -> None:
42794276
opname = "acquire shell in" if args.verb == Verb.shell else "boot"
42804277
if config.output_format not in (OutputFormat.directory, OutputFormat.disk):

‎mkosi/config.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,7 @@ def supports_cmdline(self) -> bool:
111111
Verb.box,
112112
Verb.sandbox,
113113
Verb.dependencies,
114+
Verb.latest_snapshot,
114115
)
115116

116117
def needs_tools(self) -> bool:

‎mkosi/snapshot.py‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# SPDX-License-Identifier: LGPL-2.1-or-later
2+
3+
import argparse
4+
from pathlib import Path
5+
6+
from mkosi.config import Args, Config
7+
from mkosi.run import run
8+
9+
10+
def update_snapshot(path: Path, snapshot: str) -> bool:
11+
lines = path.read_text().splitlines()
12+
13+
# Check if there's already a Snapshot= line we can replace.
14+
found = False
15+
new = []
16+
for line in lines:
17+
if line.startswith("Snapshot="):
18+
if line == f"Snapshot={snapshot}":
19+
# Already up to date.
20+
return False
21+
new.append(f"Snapshot={snapshot}")
22+
found = True
23+
else:
24+
new.append(line)
25+
26+
if not found:
27+
# Add Snapshot= after the last key in the [Distribution] section.
28+
result: list[str] = []
29+
added = False
30+
in_distribution = False
31+
for i, line in enumerate(new):
32+
result.append(line)
33+
if line.strip() == "[Distribution]":
34+
in_distribution = True
35+
continue
36+
if in_distribution:
37+
# Check if the next line starts a new section or if we're at the last line of the section.
38+
next_is_end = (i + 1 >= len(new)) or new[i + 1].startswith("[")
39+
is_blank = line.strip() == ""
40+
if next_is_end and not is_blank:
41+
result.append(f"Snapshot={snapshot}")
42+
added = True
43+
in_distribution = False
44+
elif next_is_end and is_blank:
45+
# Insert before the blank line.
46+
result.pop()
47+
result.append(f"Snapshot={snapshot}")
48+
result.append(line)
49+
added = True
50+
in_distribution = False
51+
52+
if not added:
53+
# No [Distribution] section exists; add one.
54+
result = new + ["", "[Distribution]", f"Snapshot={snapshot}"]
55+
56+
new = result
57+
58+
path.write_text("\n".join(new) + "\n")
59+
return True
60+
61+
62+
def run_latest_snapshot(args: Args, config: Config) -> None:
63+
p = argparse.ArgumentParser(
64+
prog="mkosi latest-snapshot",
65+
description="Fetch the latest snapshot for a distribution and optionally update a config file.",
66+
)
67+
p.add_argument("--update", metavar="PATH", type=Path, help="path to the config file to update")
68+
p.add_argument("--commit", "-c", action="store_true", default=False, help="commit the change with git")
69+
latestargs = p.parse_args(args.cmdline)
70+
71+
snapshot = config.distribution.installer.latest_snapshot(config)
72+
73+
if not latestargs.update:
74+
print(snapshot)
75+
return
76+
77+
print(f"Latest snapshot for {config.distribution}: {snapshot}")
78+
79+
if not update_snapshot(latestargs.update, snapshot):
80+
print("Snapshot already up to date, nothing to do.")
81+
return
82+
83+
print(f"Updated {latestargs.update} with Snapshot={snapshot}")
84+
85+
if latestargs.commit:
86+
msg = f"mkosi: Update {config.distribution} {config.release} snapshot to {snapshot}"
87+
88+
run(["git", "add", str(latestargs.update)])
89+
run(["git", "commit", "-m", msg])

0 commit comments

Comments
 (0)