Skip to content

README.md

Latest commit

 

History

History
206 lines (144 loc) · 11.9 KB

README.md

File metadata and controls

206 lines (144 loc) · 11.9 KB

Awesome Python Rs Awesome

Python tools, libraries, and frameworks with Rust under the hood.

Contents

Arrow & Columnar Data

  • arro3 - A minimal Python library for Apache Arrow, binding to the Rust Arrow crate.

Bioinformatics & Astronomy

  • bed-reader - Read and write the PLINK BED format, simply and efficiently.
  • haem - A Python library for working on bioinformatics problems.
  • mocpy - Astronomical Python library for describing coverage regions on the unit sphere.
  • satkit - Satellite and orbital mechanics toolkit for astrodynamics calculations including orbit propagation and coordinate transformations.

Cloud Storage

  • obstore - High-throughput Python interface to S3, GCS, and Azure Storage.
  • opendal - Apache OpenDAL: unified data access layer for all storage services.

Cryptography & Hashing

  • cryptography - The standard Python cryptographic library, with performance-critical parts in Rust.
  • blake3-py - Python bindings for the BLAKE3 cryptographic hash function.
  • johnnycanencrypt - OpenPGP library with Yubikey support.

Data Processing & DataFrames

  • polars - DataFrame library with lazy evaluation and parallel execution.
  • datafusion-python - Python bindings for Apache DataFusion, an in-memory query engine.
  • delta-rs - Native Rust library for Delta Lake with Python bindings (deltalake on PyPI).
  • connector-x - Fastest library to load data from databases into DataFrames.
  • pathway - Performant Python ETL framework with a Rust runtime.
  • hudi-rs - Native Rust implementation for Apache Hudi with Python bindings.
  • sail - Unifying stream, batch, and AI workloads with Apache Spark compatibility.

Date & Time

  • pendulum - Python datetimes made easy; performance-critical parts rewritten from C to Rust in v3.
  • whenever - Typed and DST-safe datetimes for Python.

Data Structures

  • rpds-py - Python bindings to the Rust rpds crate for persistent data structures.
  • fastbloom - A fast bloom filter and counting bloom filter.
  • fastuuid - Python bindings to Rust's UUID library.

File Watching & System Utilities

  • watchfiles - Simple, modern, fast file watching and code reload, powered by Rust's notify crate.
  • cramjam - Thin Python bindings to de/compression algorithms (snappy, brotli, lz4, zstd, etc.).

Finance

  • finalytics - Investment analysis library.
  • rateslib - A fixed income library for Python using Rust extensions.

Geospatial

  • geo-index - Packed, immutable, zero-copy spatial indexes.
  • tzfpy - Fast longitude/latitude to timezone name conversion.
  • utiles - Fast web-map tile utilities.

Graph Libraries

  • rustworkx - A high-performance Python graph library (originally created for Qiskit).

HTTP Clients

  • primp - Fast HTTP client that can impersonate browsers by mimicking TLS/JA3/HTTP2 fingerprints.
  • pyreqwest - Fast HTTP client built on Rust's reqwest library with async/sync support and full type safety.
  • rnet - Asynchronous Python HTTP client powered by Rust.

Linting, Formatting & Type Checking

  • ruff - An extremely fast Python linter and code formatter.
  • ty - An extremely fast Python type checker and language server by Astral.
  • zuban - A high-performance Python language server and type checker with PyRight-like and Mypy-compatible modes.
  • pylyzer - A fast static type checker and language server for Python, over 100x faster than Pyright.
  • pyrefly - A fast Python type checker and language server from Meta.
  • rumdl - A high-performance Markdown linter and formatter written in Rust.

Miscellaneous

  • pycrdt - Python bindings for the Yrs Rust CRDT implementation (collaborative editing).
  • pyxel - A retro game engine for Python, with core written in Rust.
  • ry - Collection of Python bindings to Rust crates providing utilities for async HTTP, datetime, file I/O, and compression.

Package Management & Tooling

  • maturin - Build and publish Rust-based Python packages with pyo3, cffi, and uniffi bindings.
  • pixi - A fast conda/pip package manager.
  • prek - A fast Rust-based reimplementation of the pre-commit framework for managing Git hooks.
  • uv - An extremely fast Python package and project manager. 10-100x faster than pip, replaces pip-tools, poetry, pyenv, pipx, and virtualenv in a single tool.

Profiling

  • filprofiler - Memory profiler for Python that identifies peak memory usage and allocation sources in data processing applications.

Rust-Python Interop

  • PyO3 - Rust bindings for the Python interpreter. The foundation for most projects on this list.
  • setuptools-rust - Setuptools plugin for Rust extensions.
  • RustPython - A Python 3 interpreter written entirely in Rust.

Scientific Computing

  • river - Online machine learning in Python; computationally heavy algorithms in Rust.
  • forust - A lightweight gradient boosted decision tree library.
  • radiate - A high-performance evolution engine for genetic programming and evolutionary algorithms.
  • cellular_raza - A cellular agent-based simulation framework.
  • feos - Lightning fast thermodynamic modeling with a fully developed Python interface.

Search & Indexing

  • ahocorasick_rs - Fast multi-pattern string search using the Aho-Corasick algorithm, 1.5-7x faster than alternatives.
  • tantivy-py - Python bindings for Tantivy, a full-text search engine library (Lucene alternative).

Serialization & Parsing

  • orjson - Fast, correct JSON library supporting dataclasses, datetimes, and numpy.
  • jiter - Fast iterable JSON parser, used by pydantic and the OpenAI Python SDK.
  • lp_parser_rs - Parser for Linear Programming files supporting IBM CPLEX, FICO Xpress, Gurobi, and Mosek formats.
  • ormsgpack - Fast MessagePack serialization/deserialization, derived from orjson.

Sorting

  • natsort-rs - Fast natural sorting library, 5-45x faster than alternatives.

Tokenization & ML

  • tokenizers - Hugging Face's fast tokenizer library with Python bindings.
  • tiktoken - A fast BPE tokenizer for use with OpenAI's models.
  • safetensors - A safe and fast format for storing and loading tensors.

Validation & Data Modeling

Visualization

  • rerun - Visualize streams of multimodal data. Built in Rust with a Python SDK.

Web Frameworks & Servers

  • granian - A Rust HTTP server for Python WSGI/ASGI/RSGI apps, built on Hyper and Tokio.
  • robyn - A super fast async Python web framework with a Rust runtime.

Web Scraping & HTML

  • css-inline - CSS inlining implemented in Rust, for fast HTML email preparation.
  • selectolax - Fast HTML5 parser with CSS selectors, using Rust's html5ever engine.
  • html2text-rs - Python library for converting HTML to markup or plain text.
  • html-py-ever - Fast HTML parsing and CSS selecting via html5ever.
  • markupever - High-performance HTML and XML parser with CSS selector support built on Rust's html5ever engine.

Footnotes

Learning Resources

Articles

Communities

Contributing

Contributions welcome! Please read the contributing guidelines first.