Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
BentoML
Light Logo Dark Logo
BentoML

Get Started

  • Hello world
  • Adaptive batching
  • Model composition
  • Async task queues
  • Packaging for deployment
  • Cloud deployment

Learn by Examples

  • Overview
  • LLM inference: vLLM
  • Agent: Function calling
  • Agent: LangGraph
  • LLM safety: ShieldGemma
  • RAG: Document ingestion and search
  • Stable Diffusion XL Turbo
  • ComfyUI
  • ControlNet
  • MLflow
  • XGBoost

Build with BentoML

  • Create online API Services
  • Define input and output types
  • Load and manage models
  • Work with GPUs
  • Call an API endpoint
  • Parallelize requests handling
  • Run distributed Services
  • Configure lifecycle hooks
  • Mount ASGI applications
  • Add a UI with Gradio
  • Observability
    • Monitoring
    • Logging
    • Metrics
    • Tracing
  • Customize error responses
  • Test API endpoints

Scale with BentoCloud

  • Deployment
    • Create Deployments
    • Configure Deployments
    • Manage Deployments
    • Call Deployment endpoints
    • Batch inference jobs
  • Scaling
    • Concurrency and autoscaling
  • Manage secrets
  • Manage API tokens
  • Develop with Codespaces
  • Administering
    • Manage users
    • Bring Your Own Cloud
    • Configure standby instances

References

  • BentoML
    • Bento and model APIs
    • BentoML SDK
    • BentoML CLI
    • Client API
    • Framework APIs
      • Diffusers
      • ONNX
      • Scikit-Learn
      • Transformers
      • Flax
      • TensorFlow
      • TorchScript
      • XGBoost
      • Picklable Model
      • PyTorch
      • LightGBM
      • MLflow
      • CatBoost
      • fast.ai
      • EasyOCR
      • Keras
      • Ray
      • Detectron
    • Configurations
    • Batch inference
    • Exceptions
    • Container APIs
    • Types
  • BentoCloud
    • Deployment details
    • BentoCloud CLI
    • BentoCloud API
Back to top
View this page
Edit this page

Framework APIs¶

Note

This page contains the API reference for all frameworks. For more information on a specific framework, please see Framework APIs

Diffusers
Diffusers
ONNX
ONNX
Scikit-Learn
Scikit-Learn
Transformers
Transformers
Flax
Flax
TensorFlow
TensorFlow
XGBoost
XGBoost
Picklable Model
Picklable Model
PyTorch
PyTorch
LightGBM
LightGBM
MLflow
MLflow
CatBoost
CatBoost
fast.ai
fast.ai
Keras
Keras
Detectron
Detectron
EasyOCR
EasyOCR
Ray
Ray
Next
Diffusers
Previous
Client API
Copyright © 2022-2024, bentoml.com
Made with Furo