Advanced Containerization#

time expected: 12 minutes

This guide describes advanced containerization options provided by BentoML:

This is an advanced feature for user to customize container environment that are not directly supported in BentoML. For basic containerizing options, see Docker Options.

Why you may need this?#

  • If you want to customize the containerization process of your Bento.

  • If you need a certain tools, configs, prebuilt binaries that is available across all your Bento generated container images.

  • A big difference with base image features is that you don’t have to setup a custom base image and then push it to a remote registry.

Custom Base Image#

If none of the provided distros work for your use case, e.g. if your infrastructure requires all docker images to be derived from the same base image with certain security fixes and libraries, you can config BentoML to use your base image instead:

    base_image: "my_custom_image:latest"

When a base_image is provided, all other docker options will be ignored, (distro, cuda_version, system_packages, python_version). bentoml containerize will build a new image on top of the base_image with the following steps:

  • setup env vars

  • run the setup_script if provided

  • install the required Python packages

  • copy over the Bento file

  • setup the entrypoint command for serving.


Warning: user must ensure that the provided base image has desired Python version installed. If the base image you have doesn’t have Python, you may install python via a setup_script. The implementation of the script depends on the base image distro or the package manager available.

    base_image: "my_custom_image:latest"
    setup_script: "./"


By default, BentoML supports multi-platform docker image build out-of-the-box. However, when a custom base_image is provided, the generated Dockerfile can only be used for building linux/amd64 platform docker images.

If you are running BentoML from an Apple M1 device or an ARM based computer, make sure to pass the --opt platform=linux/amd64 parameter when containerizing a Bento. e.g.:

bentoml containerize iris_classifier:latest --opt platform=linux/amd64

Dockerfile Template#

The dockerfile_template field gives the user full control over how the Dockerfile is generated for a Bento by extending the template used by BentoML.

First, create a Dockerfile.template file next to your bentofile.yaml build file. This file should follow the Jinja2 template language, and extend BentoML’s base template and blocks. The template should render a valid Dockerfile. For example:

{% extends bento_base_template %}
{{ super() }}
RUN echo "We are running this during bentoml containerize!"
{% endblock %}

Then add the path to your template file to the dockerfile_template field in your :code: bentofile.yaml:

    dockerfile_template: "./Dockerfile.template"

Now run bentoml build to build a new Bento. It will contain a Dockerfile generated with the custom template. To confirm the generated Dockerfile works as expected, run bentoml containerize <bento> to build a docker image with it.

View the generated Dockerfile content

During development and debugging, you may want to see the generated Dockerfile. Here’s shortcut for that:

cat "$(bentoml get <bento>:<tag> -o path)/env/docker/Dockerfile"


  1. Building TensorFlow custom op

  2. Access AWS credentials during image build

Building TensorFlow custom op#

Let’s start with an example that builds a custom TensorFlow op binary into a Bento, which is based on implementation details:

Define the following Dockerfile.template:

{% extends bento_base_template %}

{{ super() }}


COPY ./src/tfops/ .

RUN pip3 install tensorflow
RUN set -ex && \
	TF_CFLAGS=( $(python3 -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_compile_flags()))') ) && \
	TF_LFLAGS=( $(python3 -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_link_flags()))') ) && \
	g++ --std=c++14 -shared -o -fPIC ${TF_CFLAGS[@]} ${TF_LFLAGS[@]} -I$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include());') -D_GLIBCXX_USE_CXX11_ABI=0 -O2

{% endblock %}
{{ super() }}
RUN stat /usr/lib/
{% endblock %}

Then add the following to your bentofile.yaml:

  - ""
  - tensorflow
  dockerfile_template: ./Dockerfile.template

Proceed to build your Bento with bentoml build and containerize with bentoml containerize:

bentoml build

bentoml containerize <bento>:<tag>


You can also provide --progress plain to see the progress from buildkit in plain text

bentoml containerize --progress plain <bento>:<tag>

Access AWS credentials during image build#

We will now demonstrate how to provide AWS credentials to a Bento via two approaches:

  1. Using environment variables.

  2. Mount credentials from host.


Remarks: We recommend for most cases to use the second option (Mount credentials from host) as it prevents any securities leak.

By default BentoML uses the latest dockerfile frontend which allows mounting secrets to container.

For both examples, you will need to add the following to your bentofile.yaml:

  - awscli
  dockerfile_template: ./Dockerfile.template

Using environment variables#

Define the following Dockerfile.template:

{% extends bento_base_template %}
{{ super() }}


{% endblock %}
{{ super() }}

RUN aws s3 cp s3://path/to/file {{ bento__path }}

{% endblock %}

After building the bento with bentoml build, you can then pass AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID as arguments to bentoml containerize:

bentoml containerize --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
                     --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \

Mount credentials from host#

Define the following Dockerfile.template:

{% extends bento_base_template %}
{{ super() }}

RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
     aws s3 cp s3://path/to/file {{ bento__path }}

{% endblock %}

Follow the above addition to bentofile.yaml to include awscli and the custom dockerfile template.

To pass in secrets to the Bento, pass it via --secret to bentoml containerize:

bentoml containerize --secret id=aws,src=$HOME/.aws/credentials <bento>:<tag>

See also

Mounting Secrets

Writing dockerfile_template#

BentoML utilize Jinja2 to structure a Dockerfile.template.

The Dockerfile template is a mix between Jinja2 syntax and Dockerfile syntax. BentoML set both trim_blocks and lstrip_blocks in Jinja templates environment to True.


Make sure that your Dockerfile instruction is unindented as if you are writting a normal Dockerfile.

An example of a Dockerfile template takes advantage of multi-stage build to isolate the installation of a local library mypackage:

{% extends bento_base_template %}
FROM --platform=$BUILDPLATFORM python:3.7-slim as buildstage
RUN mkdir /tmp/mypackage

WORKDIR /tmp/mypackage/
COPY mypackage .
RUN python sdist && mv dist/mypackage-0.0.1.tar.gz mypackage.tar.gz

{{ super() }}
{% endblock %}
{{ super() }}
COPY --from=buildstage mypackage.tar.gz /tmp/wheels/
RUN --network=none pip install --find-links /tmp/wheels mypackage
{% endblock %}


Notice how for all Dockerfile instruction, we consider as if the Jinja logics aren’t there 🚀.

Jinja templates#

One of the powerful features Jinja offers is its template inheritance. This allows BentoML to enable users to fully customize how to structure a Bento’s Dockerfile.


To use a custom Dockerfile template, users have to provide a file with a format that follows the Jinja2 template syntax. The template file should have extensions of .j2, .template, .jinja.


This section is not meant to be a complete reference on Jinja2. For any advanced features from on Jinja2, please refers to their Templates Design Documentation.

To construct a custom Dockerfile template, users have to provide an extends block at the beginning of the Dockerfile template Dockerfile.template followed by the given base template name bento_base_template:

{% extends bento_base_template %}


Warning: If you pass in a generic Dockerfile file, and then run bentoml build to build a Bento and it doesn’t throw any errors.

However, when you try to run bentoml containerize, this won’t work.

This is an expected behaviour from Jinja2, where Jinja2 accepts any file as a template.

We decided not to put any restrictions to validate the template file, simply because we want to enable users to customize to their own needs.

{{ super() }}#

As you can notice throughout this guides, we use a special function {{ super() }}. This is a Jinja features that allow users to call content of parent block. This enables users to fully extend base templates provided by BentoML to ensure that the result Bentos can be containerized.

See also

{{ super() }} Syntax for more information on template inheritance.


BentoML defines a sets of Blocks under the object bento_base_template.

All exported blocks that users can use to extend are as follow:




Instructions to set up multi architecture supports, base images as well as installing system packages that is defined by users.


Setup bento users with correct UID, GID and directory for a 🍱.


Add users environment variables (if specified) and other required variables from BentoML.


Setup components for a 🍱 , including installing pip packages, running setup scripts, installing bentoml, etc.


Finalize ports and set ENTRYPOINT and CMD for the 🍱.


All the defined blocks are prefixed with SETUP_BENTO_*. This is to ensure that users can extend blocks defined by BentoML without sacrificing the flexibility of a Jinja template.

To extend any given block, users can do so by adding {{ super() }} at any point inside block.

Dockerfile instruction#

See also

Dockerfile reference for writing a Dockerfile.

We recommend that users should use the following Dockerfile instructions in their custom Dockerfile templates: ENV, RUN, ARG. These instructions are mostly used and often times will get the jobs done.

The use of the following instructions can be potentially harmful. They should be reserved for specialized advanced use cases.


Reasons not to use


Since the containerized Bento is a multi-stage builds container, adding FROM statement will result in failure to containerize the given Bento.


BentoML uses heredoc syntax and using bash in our containerization process. Hence changing SHELL will result in failure.


Changing CMD will inherently modify the behaviour of the bento container where docker won’t be able to run the bento inside the container. More below

The following instructions should be used with caution:


Since WORKDIR determines the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile, make sure that your instructions define the correct path to any working files.


By default, all paths for Bento-related files will be generated to its fspath, which ensures that Bento will work regardless of WORKDIR


The flexibility of a Jinja template also brings up the flexibility of setting up ENTRYPOINT and CMD.

From Dockerfile documentation:

Only the last ENTRYPOINT instruction in the Dockerfile will have an effect.

By default, a Bento sets:

ENTRYPOINT [ "{{ bento__entrypoint }}" ]

CMD ["bentoml", "serve", "{{ bento__path }}"]

This aboved instructions ensure that whenever docker run is invoked on the 🍱 container, bentoml is called correctly.

In scenarios where one needs to setup a custom ENTRYPOINT, make sure to use the ENTRYPOINT instruction under the SETUP_BENTO_ENTRYPOINT block as follows:

{% extends bento_base_template %}
{{ super() }}

ENTRYPOINT [ "{{ bento__entrypoint }}", "python", "-m", "awslambdaric" ]
{% endblock %}


{{ bento__entrypoint }} is the path the BentoML entrypoint, nothinig special here 😏.

Read more about CMD and ENTRYPOINT interaction here.

Advanced Options#

The next part goes into advanced options. Skip this part if you are not comfortable with using it.

Dockerfile variables#

BentoML does expose some variables that user can modify to fit their needs.

The following are the variables that users can set in their custom Dockerfile template:




Setup bento home, default to /home/{{ bento__user }}


Setup bento user, default to bentoml


Setup UID and GID for the user, default to 1034:1034


Setup bento path, default to /home/{{ bento__user }}/bento

If any of the aforementioned fields are set with {% set ... %}, then we will use your value instead, otherwise a default value will be used.

Adding conda to CUDA-enabled Bento#


Warning: miniconda install scripts provided by ContinuumIO (the parent company of Anaconda) supports Python 3.7 to 3.9. Make sure that you are using the correct python version under docker.python_version.

If you need to use conda for CUDA images, use the following template ( partially extracted from ContinuumIO/docker-images ):

Expands me
{% import '_macros.j2' as common %}
{% extends bento_base_template %}
{# Make sure to change the correct python_version and conda version accordingly. #}
{# example: py38_4.10.3 #}
{# refers to for miniconda3 base #}
{% set conda_version="py39_4.11.0" %}
{% set conda_path="/opt/conda" %}
{% set conda_exec=[conda_path, "bin", "conda"] | join("/") %}
FROM debian:bullseye-slim as conda-build

RUN --mount=type=cache,from=cached,sharing=shared,target=/var/cache/apt \
	--mount=type=cache,from=cached,sharing=shared,target=/var/lib/apt \
	apt-get update -y && \
	apt-get install -y --no-install-recommends --allow-remove-essential \
	software-properties-common \
	bzip2 \
	ca-certificates \
	git \
	libglib2.0-0 \
	libsm6 \
	libxext6 \
	libxrender1 \
	mercurial \
	openssh-client \
	procps \
	subversion \
	wget && \
	apt-get clean

ENV PATH {{ conda_path }}/bin:$PATH

ARG CONDA_VERSION={{ conda_version }}

RUN set -ex && \
	UNAME_M=$(uname -m) && \
	if [ "${UNAME_M}" = "x86_64" ]; then \
	SHA256SUM="4ee9c3aa53329cd7a63b49877c0babb49b19b7e5af29807b793a76bdb1d362b4"; \
	elif [ "${UNAME_M}" = "s390x" ]; then \
	SHA256SUM="e5e5e89cdcef9332fe632cd25d318cf71f681eef029a24495c713b18e66a8018"; \
	elif [ "${UNAME_M}" = "aarch64" ]; then \
	SHA256SUM="00c7127a8a8d3f4b9c2ab3391c661239d5b9a88eafe895fd0f3f2a8d9c0f4556"; \
	elif [ "${UNAME_M}" = "ppc64le" ]; then \
	SHA256SUM="8ee1f8d17ef7c8cb08a85f7d858b1cb55866c06fcf7545b98c3b82e4d0277e66"; \
	fi && \
	wget "${MINICONDA_URL}" -O -q && echo "${SHA256SUM}" > shasum && \
	if [ "${CONDA_VERSION}" != "latest" ]; then \
	sha256sum --check --status shasum; \
	fi && \
	mkdir -p /opt && \
	sh -b -p {{ conda_path }} && rm shasum && \
	find {{ conda_path }}/ -follow -type f -name '*.a' -delete && \
	find {{ conda_path }}/ -follow -type f -name '*' -delete && \
	{{ conda_exec }} clean -afy

{{ super() }}

ENV PATH {{ conda_path }}/bin:$PATH

COPY --from=conda-build {{ conda_path }} {{ conda_path }}

RUN set -ex && \
	ln -s {{ conda_path }}/etc/profile.d/ /etc/profile.d/ && \
	echo ". {{ conda_path }}/etc/profile.d/" >> ~/.bashrc && \
	echo "{{ conda_exec }} activate base" >> ~/.bashrc

{% endblock %}

SHELL [ "/bin/bash", "-eo", "pipefail", "-c" ]
{{ super() }}
{{ common.setup_conda(__python_version__, bento__path, conda_path=conda_path) }}
{% endblock %}

Containerization with different container engines#

In BentoML version 1.0.11 [1], we support different container engines aside from docker.

BentoML-generated Dockerfiles from version 1.0.11 onward will be OCI-compliant and can be built with:

To use any of the aforementioned backends, they must be installed on your system. Refer to their documentation for installation and setup.


By default, BentoML will use Docker as the container backend. To use other container engines, please set the environment variable BENTOML_CONTAINERIZE_BACKEND or pass in --backend to bentoml containerize:

# set environment variable
BENTOML_CONTAINERIZE_BACKEND=buildah bentoml containerize pytorch-mnist

# or pass in --backend
bentoml containerize pytorch-mnist:latest --backend buildah

To build a BentoContainer in Python, you can use the Container SDK method

import bentoml

Register custom backend#

To register a new backend, there are two functions that need to be implemented:

  • arg_parser_func: a function that takes in keyword arguments that represents the builder commandline arguments and returns a list[str]:

    def arg_parser_func(
        context_path: str = ".",
        cache_from: Optional[str] = None,
    ) -> list[str]:
        if cache_from:
            args.extend(["--cache-from", cache_from])
        return args
  • health_func: a function that returns a bool to indicate if the backend is available:

    import shutil
    def health_func() -> bool:
        return shutil.which("limactl") is not None

To register a new backend, use bentoml.container.register_backend():

from bentoml.container import register_backend

   env={"DOCKER_BUILDKIT": "1"},
Backward compatibility with bentoml.bentos.containerize

Before 1.0.11, BentoML uses bentoml.bentos.containerize() to containerize Bento. This method is now deprecated and will be removed in the future.

BuildKit interop#

BentoML leverages BuildKit for a more extensive feature set. However, we recognise that BuildKit has come with a lot of friction for migration purposes as well as restrictions to use with other build tools (such as podman, buildah, kaniko).

Therefore, since BentoML version 1.0.11, BuildKit will be an opt-out. To disable BuildKit, pass DOCKER_BUILDKIT=0 to bentoml containerize, which aligns with the behaviour of docker build:

$ DOCKER_BUILDKIT=0 bentoml containerize ...


All Bento container will now be following OCI spec instead of Docker spec. The difference is that in OCI spec, there is no SHELL argument.


The generated Dockerfile included inside the Bento will be a minimal Dockerfile, which ensures compatibility among build tools. We encourage users to always use bentoml containerize.

If you wish to use the generated Dockerfile, make sure that you know what you are doing!

CLI enhancement#

To better support different backends, bentoml containerize will be more agnostic when it comes to parsing options.

One can pass in options for specific backend with --opt:

$ bentoml containerize pytorch-mnist:latest --backend buildx --opt platform=linux/arm64

--opt also accepts parsing :

$ bentoml containerize pytorch-mnist:latest --backend buildx --opt platform:linux/arm64


If you are seeing a warning message like:

'--platform=linux/arm64' is now deprecated, use the equivalent '--opt platform=linux/arm64' instead.

BentoML used to depends on Docker buildx. These options are now backward compatible with --opt. You can safely ignore this warning and use --opt to pass options for --backend=buildx.