SuiteSparse Benchmark Suite

Reproducible benchmark suite for measuring and comparing the performance of SuiteSparse sparse linear algebra kernels across three hardware architectures: x86-64, AArch64, and RISC-V. All benchmarks run natively on the target hardware — no emulation or cross-compilation.

Benchmarks reproduce real usage scenarios from four applications: Ceres Solver, SUNDIALS, SciPy, and NetworkX.

Benchmarked scenarios

Benchmark	SuiteSparse module	Application	Scenario
bench_cholmod	CHOLMOD	Ceres Solver	Repeated Cholesky factorization
bench_klu	KLU	SUNDIALS	BTF symbolic + repeated refactorization
bench_umfpack	UMFPACK	SciPy	Multifrontal LU (spsolve)
bench_graphblas	GraphBLAS	NetworkX	SpGEMM A×A on citation graph

Repository structure

suitesparse-benchmarks/
├── benchmarks/
│   ├── cholmod/        # CHOLMOD benchmark (Ceres Solver scenario)
│   ├── klu/            # KLU benchmark (SUNDIALS scenario)
│   ├── umfpack/        # UMFPACK benchmark (SciPy scenario)
│   └── graphblas/      # GraphBLAS benchmark (NetworkX scenario)
├── matrices/           # Test matrices (not tracked, download manually)
├── scripts/
│   ├── run_x86_64.sh   # Build and run natively on x86-64
│   ├── run_aarch64.sh  # Build and run natively on AArch64 (Linux or macOS)
│   └── run_riscv64.sh  # Build and run natively on RISC-V
└── results/            # Benchmark output (not tracked, generated by scripts)

General requirements

The following tools must be available regardless of the target architecture:

Tool	Minimum version	Purpose
GCC / G++	13.0	C/C++ compiler
CMake	3.16	Build system
make	any	Build automation
git	any	Cloning SuiteSparse source
curl	any	Downloading test matrix
tar	any	Extracting test matrix archive

Install on Ubuntu/Debian:

sudo apt-get install -y build-essential cmake git curl tar

Install on macOS:

brew install cmake git curl

---

Quick start

Clone the repository and follow the instructions for your architecture below.

git clone https://github.com/fatunkaz/suitesparse-benchmarks.git
cd suitesparse-benchmarks

---

x86-64

Script: scripts/run_x86_64.sh

Step 1 — Install packages

sudo apt-get install -y libsuitesparse-dev

Step 2 — Download test matrix

cd matrices
curl -L "https://suitesparse-collection-website.herokuapp.com/MM/Pajek/HEP-th-new.tar.gz" \
    -o HEP-th-new.tar.gz
tar -xzf HEP-th-new.tar.gz && rm HEP-th-new.tar.gz
cd ..

Step 3 — Run

./scripts/run_x86_64.sh

---

AArch64

Script: scripts/run_aarch64.sh

The script automatically detects whether it runs on Linux or macOS.

Linux (Ubuntu/Debian on AArch64)

Step 1 — Install packages

sudo apt-get install -y libsuitesparse-dev

Step 2 — Download test matrix

cd matrices
curl -L "https://suitesparse-collection-website.herokuapp.com/MM/Pajek/HEP-th-new.tar.gz" \
    -o HEP-th-new.tar.gz
tar -xzf HEP-th-new.tar.gz && rm HEP-th-new.tar.gz
cd ..

Step 3 — Run

./scripts/run_aarch64.sh

macOS (Apple Silicon)

Step 1 — Install via Homebrew

brew install suite-sparse 

Step 2 — Download test matrix

cd matrices
curl -L "https://suitesparse-collection-website.herokuapp.com/MM/Pajek/HEP-th-new.tar.gz" \
    -o HEP-th-new.tar.gz
tar -xzf HEP-th-new.tar.gz && rm HEP-th-new.tar.gz
cd ..

Step 3 — Run

./scripts/run_aarch64.sh

---

RISC-V

Script: scripts/run_riscv64.sh

Tested on Scaleway EM-RV1 (T-Head C910, Ubuntu 24.04).

Step 1 — Install packages

sudo apt install -y libsuitesparse-dev

The system package provides all required SuiteSparse components including GraphBLAS 7.4.0, which is compatible with the benchmark.

Step 2 — Download test matrix

cd matrices
curl -L "https://suitesparse-collection-website.herokuapp.com/MM/Pajek/HEP-th-new.tar.gz" \
    -o HEP-th-new.tar.gz
tar -xzf HEP-th-new.tar.gz && rm HEP-th-new.tar.gz
cd ..

Step 3 — Run

./scripts/run_riscv64.sh

---

Results

Results are saved to results/ as plain text files: <benchmark>_<arch>.txt (e.g. cholmod_x86_64.txt, klu_riscv64.txt).

Example result cholmod_x86_64

=== CHOLMOD benchmark (Ceres scenario) ===
Grid: 316 x 316, matrix size: 99856 x 99856
nnz (lower triangle): 298936
Symbolic analysis:    0.033607 s
Numeric factorization (20 iters): total 6.495540 s, avg 0.324777 s
Fill-in: nnz(L)=3282021, nnz(A_full)=498016, ratio=6.59
Solve:                0.011214 s
Residual ||Ax-b||/||b||: 4.22e-16
Peak RSS:             88128 KB
--- Summary ---
symbolic:  0.033607 s
numeric:   0.324777 s (avg over 20 iters)
solve:     0.011214 s
fill-in:   6.59
residual:  4.22e-16
peak RSS:  88128 KB
numeric:   20.21 MFLOP/s
solve:     585.32 MFLOP/s
bandwidth: 0.17 GB/s (numeric factorization)
arith. intensity: 0.1196 FLOP/byte (numeric)

Test matrix

The GraphBLAS benchmark uses Pajek/HEP-th-new from the SuiteSparse Matrix Collection — a citation graph of High Energy Physics papers (n=27,770, nnz=352,807).

Environment variables

Variable	Default	Description
X86_LIBS	$HOME/x86-libs	Path to x86-64 GraphBLAS 10.x library
ARM64_LIBS	$HOME/arm64-libs	Path to AArch64 GraphBLAS library (Linux only)

License

MIT License. See LICENSE for details.