inference
Architecting the Future: Deploying Omni-Modal Models via vLLM-Omni
If you're already familiar with vLLM (the high-throughput serving engine), think of vLLM-Omni as its "multisensory" sibling
Code Clean and Speed Fast: Understanding LLM Serving with Nano vLLM
Nano vLLM is a lightweight, clean-code implementation of the core ideas behind vLLM, a high-throughput LLM serving engine
Getting Started with vLLM: A Guide for Software Engineers
Think of vLLM as a highly optimized engine for your LLMs. If you've ever tried to run an LLM, you know it can be slow and use a lot of memory