Skip to content

fix: cache only texlive-fonts-extra installed files, install other LaTeX packages normally#14414

Merged
skjnldsv merged 1 commit intomasterfrom
copilot/optimize-latex-installation
Apr 22, 2026
Merged

fix: cache only texlive-fonts-extra installed files, install other LaTeX packages normally#14414
skjnldsv merged 1 commit intomasterfrom
copilot/optimize-latex-installation

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 17, 2026

Switch the LaTeX build cache from .deb files to installed files, and fix several related issues.

What changed

Previously, the CI cached the downloaded .deb packages and re-ran dpkg on every job to unpack them. This was too slow and kept hitting timeout limits.

The new approach installs texlive-fonts-extra once via apt, copies the installed files to ${{ github.workspace }}/.cache/texlive-fonts-extra, and saves that directory via actions/cache. On cache hit, a single sudo cp puts the files back in place. No dpkg unpacking on every run.

Other fixes in this PR:

HTML builds no longer wait for the LaTeX cache job. HTML builds have no LaTeX dependency, so there's no reason for them to sit idle while setup-latex-cache runs. They now start immediately.

Fixed cache path for self-hosted runners. The old path (~/.cache/texlive-fonts-extra) was broken on self-hosted runners. actions/cache encodes file paths relative to the workspace root, so ~/.cache resolved to different absolute paths depending on how many directory levels sit between $HOME and the workspace. On GitHub-hosted runners the workspace is 3 levels deep from $HOME, on self-hosted it's 4, so the same relative path unpacked to a different location. Using ${{ github.workspace }}/.cache/texlive-fonts-extra keeps everything workspace-relative and consistent across runner types.

Improved job names. Jobs now show as "Building admin_manual HTML" and "Building admin_manual PDF" instead of "Build admin_manual" / "Build PDF admin_manual".

Original prompt

Problem

The Install LaTeX from cache step in .github/workflows/sphinxbuild.yml is too slow — it hits a 5-minute timeout because dpkg -i unpacking all texlive .deb files from the apt cache takes too long. The current approach caches .deb files but still re-runs dpkg on every job.

Solution

Replace the current .deb-based cache approach with a cached installed files approach:

  1. After installing LaTeX via apt-get, copy the installed files to ~/.cache/latex-installed/ (a home-relative path safe for actions/cache)
  2. On cache hit, restore those files by copying them back to their system destinations using sudo cp
  3. This makes the actions/cache action work without permission issues (it only touches the home path), while a simple sudo cp handles the privileged restore

Implementation

Modify .github/workflows/sphinxbuild.yml:

In the setup-latex-cache job

Replace the current apt download-only + .deb cache approach with:

  • Install LaTeX fully via apt-get install (not --download-only)
  • Copy installed files to ~/.cache/latex-installed/ covering all relevant paths:
    • /usr/share/texlive~/.cache/latex-installed/texlive
    • /usr/share/texmf~/.cache/latex-installed/texmf
    • All relevant binaries from /usr/bin/ (latexmk, xelatex, pdflatex, lualatex, xindy, etc.) — use dpkg -L on each package to find the exact binaries
    • Any config files in /etc/texmf/~/.cache/latex-installed/etc-texmf
    • Perl/lib files that texlive depends on (check dpkg -L output)
  • chown -R $(id -u):$(id -g) the cache dir so actions/cache can save it
  • Cache the ~/.cache/latex-installed/ path with actions/cache/save

In the build job

Replace the current "Restore LaTeX apt cache" + "Install LaTeX from cache" steps with:

  • Restore ~/.cache/latex-installed/ via actions/cache/restore
  • A single restore step that uses sudo cp -r to put everything back in system paths
  • Handle binaries, shared data, config, and any symlinks correctly
  • Make sure the restored install is fully functional for latexmk, xelatex, pdflatex, and xindy

Key requirements

  • Use dpkg -L for all installed packages to get the complete, accurate file list — do not guess paths
  • The cache key should reflect the package set and OS version (similar to current key format)
  • The setup-latex-cache job should still use lookup-only on first check to skip re-running if cache already exists
  • Restore must be fast — the whole point is to avoid the slow dpkg unpack on every build
  • Remove the now-unnecessary timeout-minutes: 5 that was causing failures
  • Remove the old .deb cache paths and related steps entirely

The cache size may be large (~1-2GB) — that's acceptable given the speed improvement.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Is it possible to have the workflow update the runner image by installing dependencies, and then use that image in the following jobs ? Because it seems even installing the packages takes too much time/cpu

Assistant: ## Short answer: Not natively — but there's a better approach

GitHub Actions jobs always start from a fresh runner image — you can't mutate a runner in one job and pass that modified state to the next. Each job is an isolated VM.

The right solution for your use case is a custom Docker image stored in GHCR with LaTeX pre-installed. This avoids the install overhead entirely since the packages are baked into the image layer.


The real problem

Looking at the failing job, the Install LaTeX from cache step is hitting a 5-minute timeout while running sudo dpkg -i --force-depends on all the texlive .deb files. That's a lot of large packages to unpack sequentially.


Recommended fix: Pre-built Docker image on GHCR

Create a Dockerfile and a separate workflow that builds + publishes it to GHCR. Then use it as the container in your build jobs.

1. Add a Dockerfile:

FROM ubuntu:24.04

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
    python3-pil python3-pip \
    texlive-fonts-recommended latexmk \
    texlive-latex-extra texlive-latex-recommended texlive-xetex \
    texlive-fonts-extra-links texlive-fonts-extra xindy tex-gyre \
    make git \
  && rm -rf /var/lib/apt/lists/*

2. Add a workflow to build & publish the image:

name: Build Sphinx+LaTeX image

on:
  push:
    paths:
      - '.docker/sphinx-latex/**'
    branches: [master]
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      packages: write
    steps:
      - uses: actions/checkout@v4
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: d...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

Copilot AI changed the title [WIP] Optimize LaTeX installation from cache in sphinxbuild.yml fix: replace .deb-based LaTeX cache with installed-files cache to eliminate dpkg timeout Apr 17, 2026
Copilot AI requested a review from skjnldsv April 17, 2026 14:22
@skjnldsv skjnldsv force-pushed the copilot/optimize-latex-installation branch from a410c48 to 63a1e50 Compare April 17, 2026 14:54
@skjnldsv
Copy link
Copy Markdown
Member

@copilot, maybe we don't need to cache the other latex deps after all, since only the texlive-fonts-extra is the most problematic one. I think we can just install them normally on the build step. But we keep the manual install/restore of texlive-fonts-extra ?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 17, 2026

@copilot, maybe we don't need to cache the other latex deps after all, since only the texlive-fonts-extra is the most problematic one. I think we can just install them normally on the build step. But ...

Done in 0d1af66. Dropped the entire .deb apt-cache for the other packages — they're now just installed normally via apt-get install in the build step. The setup-latex-cache job is now exclusively for texlive-fonts-extra, and the build job restores its files via sudo cp -r before running the plain apt install for everything else.

Copilot AI changed the title fix: replace .deb-based LaTeX cache with installed-files cache to eliminate dpkg timeout fix: cache only texlive-fonts-extra installed files, install other LaTeX packages normally Apr 17, 2026
@skjnldsv skjnldsv force-pushed the copilot/optimize-latex-installation branch 3 times, most recently from c3090af to 2093746 Compare April 22, 2026 06:34
@skjnldsv skjnldsv marked this pull request as ready for review April 22, 2026 06:34
Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants