Overview

This section provides the tutorials for a curated list of example projects to help you learn how BentoML can be used for different scenarios. See the following lists for a complete collection of BentoML example projects. Browse through different categories to find the example that best suits your needs.

LLMs

Deploy an OpenAI-compatible LLM API service:

• Llama 3.2 90B

• Llama 3.1 70B

• Mistral 7B

• Pixtral 12B

• Phi 3 mini

Choose your LLM Inference runtime:

• vLLM (or OpenLLM)

• TensorRT-LLM

• LMDeploy

• MLC-LLM

• SGLang

• Hugging Face TGI

Advanced examples:

• Agent: Function calling

• Agent: LangGraph

• Multi-agent: CrewAI

• LLM safety: ShieldGemma

• RAG: LlamaIndex

• Phone call agent

• Multi-LLM routing

Image and video

Serve text-to-image and image-to-image models with BentoML:

• Stable Diffusion 3.5 Large Turbo

• Stable Diffusion 3 Medium

• Stable Diffusion XL Turbo

• Stable Video Diffusion

• ControlNet

• ComfyUI

• Check out the BentoDiffusion project to see more examples

Audio

Serve text-to-speech and speech-to-text models with BentoML:

• ChatTTS

• XTTS

• XTTS with a streaming endpoint

• WhisperX

• Bark

• Moshi

Computer vision

Serve computer vision models with BentoML:

• YOLO: Object detection

• ResNet: Image classification

Embeddings

Build embedding inference APIs with BentoML:

• SentenceTransformers

• CLIP

• ColPali

Custom models

Serve custom models with BentoML:

• MLflow

• XGBoost

• scikit-learn

Others

• BLIP inference API for image captioning and VQA (Visual Question Answering)

• Serving Moirai: Time-series forecasting as a service