Conversation
a410c48 to
63a1e50
Compare
|
@copilot, maybe we don't need to cache the other latex deps after all, since only the texlive-fonts-extra is the most problematic one. I think we can just install them normally on the build step. But we keep the manual install/restore of |
Done in 0d1af66. Dropped the entire |
c3090af to
2093746
Compare
4b053eb to
84b26a0
Compare
d1b4591 to
1007d43
Compare
Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
1007d43 to
1161013
Compare
Switch the LaTeX build cache from
.debfiles to installed files, and fix several related issues.What changed
Previously, the CI cached the downloaded
.debpackages and re-randpkgon every job to unpack them. This was too slow and kept hitting timeout limits.The new approach installs
texlive-fonts-extraonce via apt, copies the installed files to${{ github.workspace }}/.cache/texlive-fonts-extra, and saves that directory viaactions/cache. On cache hit, a singlesudo cpputs the files back in place. No dpkg unpacking on every run.Other fixes in this PR:
HTML builds no longer wait for the LaTeX cache job. HTML builds have no LaTeX dependency, so there's no reason for them to sit idle while
setup-latex-cacheruns. They now start immediately.Fixed cache path for self-hosted runners. The old path (
~/.cache/texlive-fonts-extra) was broken on self-hosted runners.actions/cacheencodes file paths relative to the workspace root, so~/.cacheresolved to different absolute paths depending on how many directory levels sit between$HOMEand the workspace. On GitHub-hosted runners the workspace is 3 levels deep from$HOME, on self-hosted it's 4, so the same relative path unpacked to a different location. Using${{ github.workspace }}/.cache/texlive-fonts-extrakeeps everything workspace-relative and consistent across runner types.Improved job names. Jobs now show as "Building admin_manual HTML" and "Building admin_manual PDF" instead of "Build admin_manual" / "Build PDF admin_manual".
Original prompt
Problem
The
Install LaTeX from cachestep in.github/workflows/sphinxbuild.ymlis too slow — it hits a 5-minute timeout becausedpkg -iunpacking all texlive.debfiles from the apt cache takes too long. The current approach caches.debfiles but still re-runsdpkgon every job.Solution
Replace the current
.deb-based cache approach with a cached installed files approach:apt-get, copy the installed files to~/.cache/latex-installed/(a home-relative path safe foractions/cache)sudo cpactions/cacheaction work without permission issues (it only touches the home path), while a simplesudo cphandles the privileged restoreImplementation
Modify
.github/workflows/sphinxbuild.yml:In the
setup-latex-cachejobReplace the current apt download-only +
.debcache approach with:apt-get install(not--download-only)~/.cache/latex-installed/covering all relevant paths:/usr/share/texlive→~/.cache/latex-installed/texlive/usr/share/texmf→~/.cache/latex-installed/texmf/usr/bin/(latexmk, xelatex, pdflatex, lualatex, xindy, etc.) — usedpkg -Lon each package to find the exact binaries/etc/texmf/→~/.cache/latex-installed/etc-texmfdpkg -Loutput)chown -R $(id -u):$(id -g)the cache dir soactions/cachecan save it~/.cache/latex-installed/path withactions/cache/saveIn the
buildjobReplace the current "Restore LaTeX apt cache" + "Install LaTeX from cache" steps with:
~/.cache/latex-installed/viaactions/cache/restoresudo cp -rto put everything back in system pathslatexmk,xelatex,pdflatex, andxindyKey requirements
dpkg -Lfor all installed packages to get the complete, accurate file list — do not guess pathssetup-latex-cachejob should still uselookup-onlyon first check to skip re-running if cache already existsdpkgunpack on every buildtimeout-minutes: 5that was causing failures.debcache paths and related steps entirelyThe cache size may be large (~1-2GB) — that's acceptable given the speed improvement.
The following is the prior conversation context from the user's chat exploration (may be truncated):
User: Is it possible to have the workflow update the runner image by installing dependencies, and then use that image in the following jobs ? Because it seems even installing the packages takes too much time/cpu
Assistant: ## Short answer: Not natively — but there's a better approach
GitHub Actions jobs always start from a fresh runner image — you can't mutate a runner in one job and pass that modified state to the next. Each job is an isolated VM.
The right solution for your use case is a custom Docker image stored in GHCR with LaTeX pre-installed. This avoids the install overhead entirely since the packages are baked into the image layer.
The real problem
Looking at the failing job, the
Install LaTeX from cachestep is hitting a 5-minute timeout while runningsudo dpkg -i --force-dependson all the texlive.debfiles. That's a lot of large packages to unpack sequentially.Recommended fix: Pre-built Docker image on GHCR
Create a
Dockerfileand a separate workflow that builds + publishes it to GHCR. Then use it as the container in yourbuildjobs.1. Add a
Dockerfile:2. Add a workflow to build & publish the image: