TensorRT LLM NVIDIANVIDIA TensorRT is an SDK for optimizing and accelerating deep learning inference on NVIDIA GPUs, featuring TensorRT-LLM, quantization tools, and up to 36x speedup over CPU.(0)0
Ray AI FrameworkRay is an open source Python-native framework for scaling and orchestrating distributed AI, ML, and GenAI workloads across CPUs and GPUs at any scale.(0)0
P Petals AIPetals is an open-source framework for distributed LLM inference. Run Llama 3.1 (405B), Mixtral, Falcon, and BLOOM collaboratively across a peer-to-peer GPU network using a consumer GPU or Google Colab.(0)0
Milvus AI Vector DBMilvus is an open-source vector database built for GenAI applications. Perform high-speed similarity searches and scale to tens of billions of vectors with minimal performance loss.(0)0
QuivrQuivr is an opinionated open-source RAG framework supporting any LLM (GPT-4, Groq, Llama) and vectorstore (PGVector, Faiss). Build AI-powered apps faster without reinventing retrieval pipelines.(0)0
MetaflowBuild and manage real-life ML, AI, and data science projects with Metaflow. Open-source framework with versioning, orchestration, and cloud-scale compute originally built at Netflix.(0)0
MemGPT AgentLetta (MemGPT) is an open-source platform for building stateful AI agents with advanced memory that learns and self-improves over time.(0)0
FastChatFastChat is an open-source platform for training, serving, and evaluating large language models. Powers Chatbot Arena with OpenAI-compatible APIs and a distributed multi-model serving system.(0)0
MLflowMLflow is the largest open source AI engineering platform. Debug, evaluate, monitor, and deploy AI agents, LLMs, and ML models with 30M+ monthly downloads.(0)0
Jan.aiJan.ai is a free, open-source AI assistant that runs LLMs locally on your device for complete privacy, or connects to cloud models like GPT-4, Claude, and Gemini.(0)0
