Skip to content

Add workflow and script to automatically update distribution snapshots#4208

Open
daandemeyer wants to merge 3 commits intosystemd:mainfrom
daandemeyer:snapshot
Open

Add workflow and script to automatically update distribution snapshots#4208
daandemeyer wants to merge 3 commits intosystemd:mainfrom
daandemeyer:snapshot

Conversation

@daandemeyer
Copy link
Copy Markdown
Contributor

Add a GitHub Actions workflow that runs every three days to pin each distribution to its latest snapshot. The workflow runs tools/update-snapshot.py for each distribution (arch, centos, debian, fedora, opensuse, ubuntu) which fetches the latest snapshot via "mkosi latest-snapshot", updates the corresponding mkosi config file with the new Snapshot= setting, and commits the result.

The workflow then creates a pull request for the update using github-script. Each distribution gets its own dedicated branch (update-snapshot/) so that only a single pull request is open per distribution at a time. If an existing pull request is already open, it is updated in place by force-pushing to the branch.

Auto-merge is enabled on each pull request via the GraphQL enablePullRequestAutoMerge mutation so that the snapshot update lands automatically once CI is green.

Comment thread .github/workflows/update-snapshot.yml Outdated
Comment thread tools/update-snapshot.py Outdated
@daandemeyer daandemeyer marked this pull request as draft March 7, 2026 12:55
@keszybz
Copy link
Copy Markdown
Member

keszybz commented Mar 9, 2026

Looks reasonable…


yield f"{DISTRIBUTION_GPG_KEYS_UPSTREAM}/RPM-GPG-KEY-fedora-{version + 1}-primary"
except subprocess.CalledProcessError:
except urllib.error.URLError:

Check notice

Code scanning / CodeQL

Empty except Note

'except' clause does nothing but pass and there is no explanatory comment.
@daandemeyer daandemeyer marked this pull request as ready for review March 19, 2026 13:05
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 19, 2026

Claude review of PR #4208 (829c154)

Suggestions

  • Proxy SSL context applied to all connectionsmkosi/curl.py:42 — The proxy-specific certificate settings (proxy_client_certificate, proxy_client_key) are loaded into the HTTPS handler's SSL context, which applies to all HTTPS connections, not just the proxy. The old curl CLI used --proxy-cert/--proxy-key which only affected the proxy tunnel. (Note: improved in this revision — the custom context is now only created when proxy is configured and custom certs exist, but the fundamental urllib limitation remains.)
  • No timeout on HTTP requestsmkosi/curl.py:52opener.open(url) has no timeout parameter; the default socket timeout is None (block indefinitely). The old curl CLI had built-in timeouts.
  • ${{ inputs.config }} interpolated directly in shell.github/actions/mkosi-update-snapshot/action.yml:53 — Unlike distribution and release (passed via env:), config is interpolated directly into the run: block. Pass it through an env var for consistency and to avoid injection if the path ever contains special characters.
  • Script injection risk in github-script.github/actions/mkosi-update-snapshot/action.yml:137${{ steps.update.outputs.title }} is interpolated directly into a JS string literal. If the value contained a single quote it would break syntax or allow injection. Use process.env via an env: block instead.
  • Unpinned action version.github/workflows/update-snapshot.yml:36actions/create-github-app-token@v2 uses a mutable tag. All other external action references in the repo are pinned to commit SHAs.
  • update_snapshot edge case with empty [Distribution] sectionmkosi/snapshot.py:46 — If [Distribution] is immediately followed by another section header (no keys between them), Snapshot= is inserted under the wrong section. Also, if [Distribution] is the last line with no keys, the fallback creates a duplicate section. Current workflow configs are unaffected, but adding a guard for line.startswith("[") in the in_distribution block would make this robust.

Nits

  • Missing trailing newlinetools/mkosi-manifest-diff.py:72 — File is missing a final newline, inconsistent with the rest of the repo.

All previous review items have been addressed in the latest revision. The PR looks good.

Workflow run

Comment thread mkosi/curl.py Outdated
Comment thread mkosi/curl.py Outdated
Comment thread .github/actions/mkosi-update-snapshot/action.yml Outdated
Comment thread .github/actions/mkosi-update-snapshot/action.yml Outdated
Comment thread .github/workflows/update-snapshot.yml Outdated
Comment thread mkosi/snapshot.py
Comment thread tools/mkosi-manifest-diff.py Outdated
Comment thread mkosi/curl.py Outdated
Rename to fetch while we're at it.
Add a GitHub Actions workflow that runs every three days to pin each
distribution to its latest snapshot. To make this work, we extend the
latest-snapshot verb with the option to write the snapshot to a config
file provided by the user. We do this to avoid having to start tracking
which config file provided a setting. Even if we had that, we still would
need the explicit config file to bootstrap the initial write so we might
as well insist on the user providing it. The workflow runs mkosi
latest-snapshot for each distribution (arch, centos, debian, fedora,
opensuse, ubuntu) which fetches the latest snapshot, updates the
corresponding mkosi config file with the new Snapshot= setting, and commits
the result.

The workflow then creates a pull request for the update using
github-script. Each distribution gets its own dedicated branch
(update-snapshot/<distro>) so that only a single pull request is open
per distribution at a time. If an existing pull request is already
open, it is updated in place by force-pushing to the branch.

Auto-merge is enabled on each pull request via the GraphQL
enablePullRequestAutoMerge mutation so that the snapshot update lands
automatically once CI is green.
Comment on lines +24 to +25
old_keys = set(old_packages.keys())
new_keys = set(new_packages.keys())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The set() call around them is not really necessary, dict_keys already support all set operations by themselves.

Comment on lines +61 to +68
counts = []
if changed:
counts.append(f"{len(changed)} upgraded")
if added:
counts.append(f"{len(added)} added")
if removed:
counts.append(f"{len(removed)} removed")
print(f"**Summary:** {', '.join(counts)}")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: You could just go with

print(f"**Summary:** {len(changed)} upgraded, {len(added)} added, {len(removed)} removed")

This way the last line always has the same structure, which, at least I, find visually easier when comparing output.

branch="update-snapshot/${DISTRIBUTION:-default}"
git checkout -B "$branch"

mkosi ${DISTRIBUTION:+-d "$DISTRIBUTION"} ${RELEASE:+-r "$RELEASE"} latest-snapshot -- --update "$CONFIG" --commit
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest-snapshot command is not documented. I think it should be, since it's quite useful for the snapshot functionality.

It's also a bit of a break with the rest of the config parsing, that this implements it's own argument parser. That should be documented. It seems a bit peculiar, since so far every subcommand's options just went into the big option bag.

Comment thread mkosi/snapshot.py
Comment on lines +60 to +62
if not added:
# No [Distribution] section exists; add one.
result = new + ["", "[Distribution]", f"Snapshot={snapshot}"]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has the edge case, that if there is just an empty distribution section

[Distribution]

add the end of the file, it will add a second distribution section. So adding the section header should be contingent on not being in_distribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

4 participants