vLLMvLLM is an open-source high-throughput LLM inference library supporting GPU, CPU, and TPU backends with an OpenAI-compatible API, PagedAttention, and production deployment tools.(0)AI Models & Infrastructure·LLM Developer Tools·AI Frameworks0
Unleash AI Feature FlagUnleash is a private, secure, open-source feature management platform with feature flags, kill switches, and auditability built for enterprise scale.(0)DevOps Tools·Workflow Automation Tools·AI Infrastructure Tools0
TensorFlow LiteLiteRT (formerly TensorFlow Lite) is Google's open-source framework for deploying ML and GenAI models on Android, iOS, web, desktop, and IoT devices with GPU/NPU acceleration.(0)AI Models & Infrastructure·LLM Developer Tools·AI Frameworks0
ONNX RuntimeONNX Runtime is Microsoft's open-source AI engine for accelerated machine learning inference and training across cloud, edge, mobile, and web platforms.(0)AI Models & Infrastructure·LLM Developer Tools·AI Frameworks0
LocalAILocalAI is a free, open-source alternative to OpenAI and Anthropic. Run LLMs, image generation, audio, and autonomous agents locally on your own hardware with complete privacy.(0)LLM Developer Tools·AI Infrastructure Tools·AI Frameworks0
llama.cppRun large language models locally with llama.cpp — a high-performance, open-source C/C++ inference engine supporting CUDA, Metal, Vulkan, and GGUF quantization for 50+ model architectures.(0)Foundation Models·LLM Developer Tools·AI Infrastructure Tools0
Ray AI FrameworkRay is an open source Python-native framework for scaling and orchestrating distributed AI, ML, and GenAI workloads across CPUs and GPUs at any scale.(0)AI Models & Infrastructure·AI Infrastructure Tools·AI Frameworks0
QuivrQuivr is an opinionated open-source RAG framework supporting any LLM (GPT-4, Groq, Llama) and vectorstore (PGVector, Faiss). Build AI-powered apps faster without reinventing retrieval pipelines.(0)Knowledge Base Bots·LLM Developer Tools·AI Frameworks0
QdrantQdrant is a high-performance, open-source vector search engine and database written in Rust. Build production-ready RAG pipelines, recommendation systems, and semantic search at scale.(0)Analytics Databases·LLM Developer Tools·AI Infrastructure Tools0
MemGPT AgentLetta (MemGPT) is an open-source platform for building stateful AI agents with advanced memory that learns and self-improves over time.(0)AI Agents·LLM Developer Tools·AI Frameworks0
