Bento build options¶
Build options refer to a set of configurations for building a BentoML project into a Bento. These options can be defined in a pyproject.toml
file under the [tool.bentoml.build]
section or a YAML file (typically named bentofile.yaml
).
Available fields¶
service
¶
service
is a required field and points to where a Service object resides. It is often defined as service: "service:class-name"
.
service
: The Python module, namely theservice.py
file.class-name
: The class-based Service’s name created inservice.py
, decorated with@bentoml.service
. If you have multiple Services inservice.py
, you can specify the main Service receiving user requests in the configuration. Other Services will be started together with this main Service.
[tool.bentoml.build]
service = "service:MyService"
service: "service:MyService"
description
¶
description
allows you to annotate your Bento with relevant documentation, which can be written in plain text or Markdown format. You can either provide the description inline or reference an external file.
To define the description inline:
[tool.bentoml.build]
service = "service:svc"
description = "This is an inline description for the Service. BentoML is awesome!"
To define the description using a file:
[tool.bentoml.build]
service = "service:svc"
description = "file: ./README.md"
To define the description inline:
service: "service:svc"
description: |
## Description For My Bento 🍱
Use **any markdown syntax** here!
> BentoML is awesome!
To define the description using a file:
service: "service:svc"
description: "file: ./README.md"
For descriptions sourced from an external file, you can use either an absolute or relative path. Make sure the file exists at the specified path when the bentoml build
command is run. For relative paths, the reference point is the build_ctx
, which defaults to the directory from which bentoml build
is executed.
labels
¶
labels
are key-value pairs associated with objects. In BentoML, both Bentos and models can have labels attached to them. These labels can serve various purposes, such as identifying or categorizing Bentos and models in BentoCloud. You can add or modify labels at any time.
[tool.bentoml.build.labels]
owner = "bentoml-team"
stage = "not-ready"
labels:
owner: bentoml-team
stage: not-ready
include
¶
You use the include
field to include specific files when building the Bento. It supports wildcard characters and directory pattern matching. For example, setting it to *.py
means every Python file under the existing build_ctx
will be packaged into the Bento.
[tool.bentoml.build]
include = ["data/", "**/*.py", "config/*.json", "path/to/a/file.csv"]
include:
- "data/"
- "**/*.py"
- "config/*.json"
- "path/to/a/file.csv"
If this field is not specified, BentoML includes all files under the build_ctx
by default, excluding those explicitly set in the exclude
field.
Note
Both include
and exclude
fields support gitignore style pattern matching.
exclude
¶
You use the exclude
field to exclude specific files when building the Bento. This is useful when you have many files in the working directory, as you only need to
specify the files to be ignored.
When setting this field, you specify the file pathspecs (similar to .gitignore
) that are relative to the build_ctx
directory.
[tool.bentoml.build]
include = ["data/", "**/*.py"]
exclude = ["tests/", "secrets.key"]
include:
- "data/"
- "**/*.py"
exclude:
- "tests/"
- "secrets.key"
Alternatively, create a .bentoignore
file in the build_ctx
directory as follows:
__pycache__/
*.py[cod]
*$py.class
.ipynb_checkpoints/
training_data/
Note
exclude
is always applied after include
.
models
¶
You can specify the model to be used for building a Bento using a string model tag or a dictionary. If you don’t define models in the configuration, the model specified in the Service is used to build the Bento.
[tool.bentoml.build]
models = [
"summarization-model:latest",
{ tag = "summarization-model:version1", filter = "label:staging", alias = "summarization-model_v1" }
]
models:
- "summarization-model:latest" # A string model tag
- tag: "summarization-model:version1" # A dictionary
filter: "label:staging"
alias: "summarization-model_v1"
tag
: The name and version of the model, separated by a colon.filter
: This field uses the same filter syntax in BentoCloud. You use a filter to list specific models, such as the models with the same label. You can add multiple comma-separated filters to a model.alias
: An alias for the model. If this is specified, you can use it directly in code likebentoml.models.get(alias)
.
Python packages¶
You specify the required Python packages for a given Bento using the python.packages
field. BentoML allows you to specify the
desired version and install a package from a custom PyPI source or from a GitHub repository. If a package lacks a specific version,
BentoML will lock the versions of all Python packages for the current platform and Python when building a Bento.
When using a pyproject.toml
file, you can define the required Python packages through project.dependencies
:
[project]
dependencies = [
"numpy",
"matplotlib==3.5.1",
"package>=0.2,<0.3",
"torchvision==0.9.2",
"git+https://github.com/username/mylib.git@main",
]
This is equivalent to using python.packages
:
[tool.bentoml.build.python]
packages = [
"numpy",
"matplotlib==3.5.1",
"package>=0.2,<0.3",
"torchvision==0.9.2",
"git+https://github.com/username/mylib.git@main"
]
If you specify both project.dependencies
and tool.bentoml.build.python.packages
, the dependencies are combined together.
python:
packages:
- "numpy"
- "matplotlib==3.5.1"
- "package>=0.2,<0.3"
- "torchvision==0.9.2"
- "git+https://github.com/username/mylib.git@main"
Note
You don’t need to specify bentoml
as a dependency in this field since the current version of BentoML will be added to the list by default. However,
you can override this by specifying a different BentoML version.
To include a package from a GitHub repository, use the pip requirements file format. You can specify the repository URL, the branch, tag, or commit to install from, and the subdirectory if the Python package is not in the root of the repository.
[tool.bentoml.build.python]
packages = [
"git+https://github.com/username/repository.git@branch_name",
"git+https://github.com/username/repository.git@v1.0.0",
"git+https://github.com/username/repository.git@abcdef1234567890abcdef1234567890abcdef12",
"git+https://github.com/username/repository.git@branch_name#subdirectory=package_dir",
]
python:
packages:
- "git+https://github.com/username/repository.git@branch_name"
- "git+https://github.com/username/repository.git@v1.0.0"
- "git+https://github.com/username/repository.git@abcdef1234567890abcdef1234567890abcdef12"
- "git+https://github.com/username/repository.git@branch_name#subdirectory=package_dir"
If your project depends on a private GitHub repository, you can include the Python package from the repository via SSH. Make sure that the environment where BentoML is running has the appropriate SSH keys configured and that these keys are added to GitHub. In the following example, git@github.com:username/repository.git
is the SSH URL for the repository.
[tool.bentoml.build.python]
packages = [
"git+ssh://git@github.com/username/repository.git@branch_name"
]
python:
packages:
- "git+ssh://git@github.com/username/repository.git@branch_name"
If you already have a requirements.txt
file that defines Python packages for your project, you may also supply a path to the requirements.txt
file directly:
[tool.bentoml.build.python]
requirements_txt = "./project-a/ml-requirements.txt"
python:
requirements_txt: "./project-a/ml-requirements.txt"
Pip install options¶
You can provide additional pip install
arguments in the python
field. If provided, these arguments will be applied to all packages defined in python.packages
as
well as the requirements_txt
file.
[tool.bentoml.build.python]
requirements_txt = "./requirements.txt"
index_url = "https://my.mirror.com/simple"
no_index = false
trusted_host = ["pypi.python.org", "my.mirror.com"]
find_links = ["https://download.pytorch.org/whl/cu80/stable.html"]
extra_index_url = [
"https://<other api token>:@my.mirror.com/pypi/simple",
"https://pypi.python.org/simple"
]
pip_args = "--pre -U --force-reinstall"
python:
requirements_txt: "./requirements.txt"
index_url: "https://my.mirror.com/simple"
no_index: false
trusted_host:
- "pypi.python.org"
- "my.mirror.com"
find_links:
- "https://download.pytorch.org/whl/cu80/stable.html"
extra_index_url:
- "https://<other api token>:@my.mirror.com/pypi/simple"
- "https://pypi.python.org/simple"
pip_args: "--pre -U --force-reinstall"
Note
By default, BentoML caches pip artifacts across all local image builds to speed up the build process.
If you want to force a re-download instead of using the cache, you can specify the pip_args: "--no-cache-dir"
option in your
bentofile.yaml
file, or use the --no-cache
option in the bentoml containerize
command. For example:
$ bentoml containerize my_bento:latest --no-cache
PyPI package locking¶
By default, BentoML automatically locks all package versions, as well as all packages in
their dependency graph, and
generates a requirements.lock.txt
file. This process uses
pip-compile under the hood.
If you have already specified a version for all packages, you can optionally disable
this behavior by setting the lock_packages
field to false
:
[tool.bentoml.build.python]
requirements_txt = "./requirements.txt"
lock_packages = false
python:
requirements_txt: "./requirements.txt"
lock_packages: false
When including Python packages from GitHub repositories, use the pack_git_packages
option (it defaults to true
) to control whether these packages should be cloned and packaged during the build process. This is useful for dependencies that may not be available via standard PyPI sources or for ensuring consistency with specific versions (for example, tags and commits) of a dependency directly from a Git repository.
[tool.bentoml.build.python]
pack_git_packages = true
packages = ["git+https://github.com/username/repository.git@abcdef1234567890abcdef1234567890abcdef12"]
python:
pack_git_packages: true # Enable packaging of Git-based packages
packages:
- "git+https://github.com/username/repository.git@abcdef1234567890abcdef1234567890abcdef12"
Note that lock_packages
controls whether the versions of all dependencies, not just those from Git, are pinned at the time of building the Bento. Disabling pack_git_packages
will also disable package locking (lock_packages
) unless explicitly set.
Note
BentoML will always try to lock the package versions against Linux x86_64 platform to match the deployment target. If the bento contains dependencies or transitive dependencies with environment markers, they will be resolved against Linux x86_64 platform.
For example, if the bento requires torch
, nvidia-*
packages will also be picked up into the final lock result although they are only required for Linux x86_64 platform.
If you want to build a bento for a different platform, you can pass --platform
option to bentoml build
command with the name of the target platform. For example:
$ bentoml build --platform macos
Python wheels¶
Python .whl
files are also supported as a type of dependency to include in a
Bento. Simply provide a path to your .whl
files under the wheels
field.
[tool.bentoml.build.python]
wheels = ["./lib/my_package.whl"]
python:
wheels:
- ./lib/my_package.whl
If the wheel is hosted on a local network without TLS, you can indicate
that the domain is safe to pip with the trusted_host
field.
Python options table¶
The following table provides a full list of available configurations for the python
field.
Field |
Description |
---|---|
requirements_txt |
The path to a custom |
packages |
Packages to include in this Bento |
lock_packages |
Whether to lock the packages |
index_url |
Inputs for the |
no_index |
Whether to include the |
trusted_host |
List of trusted hosts used as inputs using the |
find_links |
List of links to find as inputs using the |
extra_index_url |
List of extra index URLs as inputs using the |
pip_args |
Any additional pip arguments that you want to add when installing a package |
wheels |
List of paths to wheels to include in the Bento |
envs
¶
Environment variables are important for managing configuration and secrets in a secure and flexible manner. They allow you to configure BentoML Services without hard-coding sensitive information, such as API keys, database credentials, or configurable parameters that might change between different environments.
You set environment variables under the envs
key in bentofile.yaml
. Each environment variable is defined with name
and value
keys. For example:
[tool.bentoml.build]
envs = [
{ name = "VAR_NAME", value = "value" },
{ name = "API_KEY", value = "your_api_key_here" }
]
envs:
- name: "VAR_NAME"
value: "value"
- name: "API_KEY"
value: "your_api_key_here"
The specified environment variables will be injected into the Bento container.
Note
If you deploy your BentoML Service on BentoCloud, you can either set environment variables through envs
in configuration or the --env
flag when running bentoml deploy
. See Environment variables for details.
conda
¶
Conda dependencies can be specified under the conda
field. For example:
[tool.bentoml.build.conda]
channels = ["default"]
dependencies = ["h2o"]
pip = ["scikit-learn==1.2.0"]
conda:
channels:
- default
dependencies:
- h2o
pip:
- "scikit-learn==1.2.0"
channels
: Custom conda channels to use. If it is not specified, BentoML will use the community-maintainedconda-forge
channel as the default.dependencies
: Custom conda dependencies to include in the environment.pip
: The specificpip
conda dependencies to include.
Optionally, you can export all dependencies from a pre-existing conda environment to an environment.yml
file, and provide this file in your pyproject.toml
or bentofile.yaml
file. If it is specified, this file will overwrite any additional option specified.
To export a conda environment:
conda env export > environment.yml
To add it in your configuration:
[tool.bentoml.build.conda]
environment_yml = "./environment.yml"
conda:
environment_yml: "./environment.yml"
Note
Unlike Python packages, BentoML does not support locking conda package versions automatically. We recommend you specify a version in the configuration file.
See also
When conda
options are provided, BentoML will select a Docker base image
that comes with Miniconda pre-installed in the generated Dockerfile. Note that only
the debian
and alpine
distro support conda
. Learn more in
the docker
section below.
docker
¶
BentoML makes it easy to deploy a Bento to a Docker container. It provides a set of options for customizing the Docker image generated from a Bento.
The following docker
field contains some basic Docker configurations:
[tool.bentoml.build.docker]
distro = "debian"
python_version = "3.11"
system_packages = ["libblas-dev", "liblapack-dev", "gfortran"]
docker:
distro: debian
python_version: "3.11"
system_packages:
- libblas-dev
- liblapack-dev
- gfortran
BentoML uses BuildKit, a cache-efficient builder toolkit, to containerize Bentos. BuildKit comes with Docker 18.09. This means if you are using Docker via Docker Desktop, BuildKit will be available by default. If you are using a standalone version of Docker, you can install BuildKit by following the instructions here.
The following sections provide detailed explanations of certain Docker configurations.
OS distros¶
The following OS distros are currently supported in BentoML:
debian
: The default value, similar to Ubuntualpine
: A minimal Docker image based on Alpine Linuxubi8
: Red Hat Universal Base Imageamazonlinux
: Amazon Linux 2
Some of the distros may not support using conda or specifying CUDA for GPU. Here is the support matrix for all distros:
Distro |
Available Python Versions |
Conda Support |
CUDA Support (GPU) |
---|---|---|---|
debian |
3.7, 3.8, 3.9, 3.10 |
Yes |
Yes |
alpine |
3.7, 3.8, 3.9, 3.10 |
Yes |
No |
ubi8 |
3.8, 3.9 |
No |
Yes |
amazonlinux |
3.7, 3.8 |
No |
No |
Setup script¶
For advanced Docker customization, you can also use the setup_script
field to inject
any script during the image build process. For example, with NLP
projects you can pre-download NLTK data in the image by setting the following values.
[tool.bentoml.build.python]
packages = ["nltk"]
[tool.bentoml.build.docker]
setup_script = "./setup.sh"
python:
packages:
- "nltk"
docker:
setup_script: "./setup.sh"
In the setup.sh
file:
#!/bin/bash
set -euxo pipefail
echo "Downloading NLTK data.."
python -m nltk.downloader all
Build a new Bento and then run bentoml containerize MY_BENTO --progress plain
to
view the Docker image build progress. The newly built Docker image will contain the
pre-downloaded NLTK dataset.
Tip
When working with bash scripts, we recommend you add set -euxo pipefail
to the beginning. Especially when set -e is missing, the script will fail silently
without raising an exception during bentoml containerize
. Learn more about
Bash Set builtin.
It is also possible to provide a Python script for initializing the Docker image. Here’s an example:
[tool.bentoml.build.python]
packages = ["nltk"]
[tool.bentoml.build.docker]
setup_script = "./setup.py"
python:
packages:
- "nltk"
docker:
setup_script: "./setup.py"
In the setup.py
file:
#!/usr/bin/env python
import nltk
print("Downloading NLTK data..")
nltk.download('treebank')
Note
Pay attention to #!/bin/bash
and #!/usr/bin/env python
in the
first line of the example scripts above. They are known as Shebang
and they are required in a setup script provided to BentoML.
Setup scripts are always executed after the specified Python packages, conda dependencies, and system packages are installed. Therefore, you can import and utilize those libraries in your setup script for the initialization process.
Docker options table¶
The following table provides a full list of available configurations for the docker
field.
Field |
Description |
---|---|
distro |
The OS distribution on the Docker image. It defaults to |
python_version |
The Python version on the Docker image. It defaults to the Python version in the build environment. |
cuda_version |
Deprecated. The CUDA version on the Docker image for running models that require GPUs. When using PyTorch or TensorFlow to run models on GPUs, we recommend you directly install them along with their respective CUDA dependencies, using |
system_packages |
The system packages that will be installed in the container. |
setup_script |
A Python or Shell script that will be executed during the Docker build process. |
base_image |
A user-provided Docker base image. This will override all other custom attributes of the image. |
dockerfile_template |
Customize the generated Dockerfile by providing a Jinja2 template that extends the default Dockerfile. |
Custom build context¶
For projects that are part of a larger codebase and interact with other local Python
modules or those containing multiple Bentos/Services, it might not be possible to
put all Service definition code and pyproject.toml
(or bentofile.yaml
) in the project’s root directory.
BentoML allows the placement of the Service definition and pyproject.toml
(or bentofile.yaml
) anywhere in the project directory.
In such scenarios, specify the build_ctx
and bentofile
arguments when running the bentoml build
command.
build_ctx
: The build context represents the working directory of your Python project. It will be prepended to the PYTHONPATH during build process, ensuring the correct import of local Python modules. By default, it’s set to the current directory where thebentoml build
command is executed.bentofile
: It defaults to thepyproject.toml
(orbentofile.yaml
) file in the build context.
To customize their values, use the following:
bentoml build -f ./src/my_project_a/bento_fraud_detect.yaml ./src/
Structure¶
By default, all created Bentos are stored in the BentoML Bento Store, which is essentially a local directory. You can go to a specific Bento directory by running the following command:
cd $(bentoml get BENTO_TAG -o path)
Inside the directory, you might see different files and sub-directories depending on the configurations in pyproject.toml
(or bentofile.yaml
). A typical Bento contains the following key sub-directories:
src
: Contains files specified in theinclude
field. These files are relative to user Python code’s CWD (current working directory), which makes importing relative modules and file paths inside user code possible.apis
: Contains API definitions auto-generated from the Service’s API specifications.env
: Contains environment-related files for Bento initialization. These files are generated based on the build options specified inpyproject.toml
(orbentofile.yaml
).
Warning
We do not recommend you change files in a Bento directly, unless it’s for debugging purposes.