Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

Android inference engine running 20B+ parameter LLMs on 4GB-8GB RAM devices. Features proprietary Layer-by-Layer (LBL) streaming, zero-copy mmap loading, and native C++/Kotlin architecture.

Convert and quantize llm models

A simple Gradio app for local translation using the GGUF versions of MADLAD-400

Privacy-first Local RAG Server: Chat with PDF & DOCX using GGUF models via llama.cpp and Qdrant. A lightweight, standalone FastAPI server with a clean HTML UI. High-performance, fully offline document intelligence. No Ollama, no cloud, no API keys.

Splinter is a lock-free shared memory bus that puts your AI governor in the same room (or even NUMA lane) as your model and inference. It even *includes* socket-free inference sidecars with more powerful tools that teach you how to build with it. Splinter fits in the size of most modern CPU instruction caches (876 ELOC W/Excellent Test Coverage) .

Emotica AI is a compassionate and therapeutic virtual assistant designed to provide empathetic and supportive conversations. It integrates a local LLaMA model for text generation, a vision model for image captioning, a RAG system for information retrieval, and emotion detection to tailor its responses.

Containerized LLM for any use-case big or small

Nectar-X-Studio is a powerful, Local AI-Inferencing application that allows the user download, create, run agents and run large language models on their own machine. With no internet connection required, Nectar ensures privacy-first, high-performance inference using cutting-edge open-source models from Hugging Face, Ollama, and beyond.

GitBot is a local Git-based platform designed to mirror the complete GitHub experience while keeping your data entirely on your own machine

AI tool to help users research using local LLMs and automated web search.

GGUF file format for dotnet