Bento and model APIs¶

This page contains management APIs for Bentos and models.

Manage Bentos¶

bentoml.list(tag: Tag | str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → t.List[Bento][source]¶

bentoml.get(tag: Tag | str, *, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]¶

bentoml.delete(tag: Tag | str, *, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>)[source]¶

Export a bento.

To export a bento to S3, you must install BentoML with extras aws:

» pip install bentoml[aws]

Examples:

# exports 'my_bento' to '/path/to/folder/my_bento-version.bento' in BentoML's default format
bentoml.export_bento('my_bento:latest', '/path/to/folder')
# note that folders can only be passed if exporting to the local filesystem; otherwise the
# full path, including the desired filename, must be passed

# exports 'my_bento' to '/path/to/folder/my_bento.bento' in BentoML's default format
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento')
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.bento')

# exports 'my_bento' to '/path/to/folder/my_bento.tar.gz' in gzip format
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.tar.gz')
# outputs a gzipped tarfile as 'my_bento.ext'
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.ext', 'gz')

# exports 'my_bento' to '/path/to/folder/my_bento/' as a folder
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento', 'folder')

# exports 'my_bento' to the S3 bucket 'my_bucket' as 'folder/my_bento-version.bento'
bentoml.export_bento('my_bento:latest', 's3://my_bucket/folder')
bentoml.export_bento('my_bento:latest', 'my_bucket/folder', protocol='s3')
bentoml.export_bento('my_bento:latest', 'my_bucket', protocol='s3', subpath='folder')
bentoml.export_bento('my_bento:latest', 'my_bucket', protocol='s3', subpath='folder',
                     user='<AWS access key>', passwd='<AWS secret key>',
                     params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the Bento to export
path – can be either: * a folder on the local filesystem * an FS URL. For example, 's3://my_bucket/folder/my_bento.bento'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the bento should be exported to
_bento_store – save Bento created to this BentoStore

Returns:

A representation of the path that the Bento was exported to. If it was exported to the local filesystem,: this will be the OS path to the exported Bento. Otherwise, it will be an FS URL.

Return type:

str

bentoml.import_bento(path: str, input_format: str | None = None, *, protocol: str | None = None, user: str | None = None, passwd: str | None = None, params: t.Optional[t.Dict[str, str]] = None, subpath: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]¶

Import a bento.

Examples:

# imports 'my_bento' from '/path/to/folder/my_bento.bento'
bentoml.import_bento('/path/to/folder/my_bento.bento')

# imports 'my_bento' from '/path/to/folder/my_bento.tar.gz'
# currently supported formats are tar.gz ('gz'),
# tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.import_bento('/path/to/folder/my_bento.tar.gz')
# treats 'my_bento.ext' as a gzipped tarfile
bentoml.import_bento('/path/to/folder/my_bento.ext', 'gz')

# imports 'my_bento', which is stored as an
# uncompressed folder, from '/path/to/folder/my_bento/'
bentoml.import_bento('/path/to/folder/my_bento', 'folder')

# imports 'my_bento' from the S3 bucket 'my_bucket',
# path 'folder/my_bento.bento'
# requires `fs-s3fs <https://pypi.org/project/fs-s3fs/>`_
bentoml.import_bento('s3://my_bucket/folder/my_bento.bento')
bentoml.import_bento('my_bucket/folder/my_bento.bento', protocol='s3')
bentoml.import_bento('my_bucket', protocol='s3',
                     subpath='folder/my_bento.bento')
bentoml.import_bento('my_bucket', protocol='s3',
                     subpath='folder/my_bento.bento',
                     user='<AWS access key>', passwd='<AWS secret key>',
                     params={'acl': 'public-read',
                             'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd,

params, and subpath) mean, see the

FS URL documentation.

Parameters:

tag – the tag of the bento to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL,

for example 's3://my_bucket/folder/my_bento.bento'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the bento should be exported to
_bento_store – the bento store to save the bento to

Returns:

the imported bento

Return type:

Bento

User-facing API for building a Bento. The available build options are identical to the keys of a valid ‘bentofile.yaml’ file.

This API will not respect any ‘bentofile.yaml’ files. Build options should instead be provided via function call parameters.

Parameters:

service – import str for finding the bentoml.Service instance build target
labels – optional immutable labels for carrying contextual info
description – optional description string in markdown format
include – list of file paths and patterns specifying files to include in Bento, default is all files under build_ctx, beside the ones excluded from the exclude parameter or a .bentoignore file for a given directory
exclude – list of file paths and patterns to exclude from the final Bento archive
docker – dictionary for configuring Bento’s containerization process, see details in bentoml._internal.bento.build_config.DockerOptions
python – dictionary for configuring Bento’s python dependencies, see details in bentoml._internal.bento.build_config.PythonOptions
conda – dictionary for configuring Bento’s conda dependencies, see details in bentoml._internal.bento.build_config.CondaOptions
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
platform – Platform to build for

Returns:

a Bento instance representing the materialized Bento saved in BentoStore

Return type:

Bento

Example

import bentoml

bentoml.build(
    service="fraud_detector.py:svc",
    version="any_version_label",  # override default version generator
    description=open("README.md").read(),
    include=['*'],
    exclude=[], # files to exclude can also be specified with a .bentoignore file
    labels={
        "foo": "bar",
        "team": "abc"
    },
    python=dict(
        packages=["tensorflow", "numpy"],
        # requirements_txt="./requirements.txt",
        index_url="http://<api token>:@mycompany.com/pypi/simple",
        trusted_host=["mycompany.com"],
        find_links=['thirdparty..'],
        extra_index_url=["..."],
        pip_args="ANY ADDITIONAL PIP INSTALL ARGS",
        wheels=["./wheels/*"],
        lock_packages=True,
    ),
    docker=dict(
        distro="amazonlinux2",
        setup_script="setup_docker_container.sh",
        python_version="3.8",
    ),
)

bentoml.bentos.build_bentofile(bentofile: str | None = None, *, version: str | None = None, labels: dict[str, str] | None = None, build_ctx: str | None = None, platform: str | None = None, bare: bool = False, reload: bool = False, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]¶

Build a Bento base on options specified in a bentofile.yaml file.

By default, this function will look for a bentofile.yaml file in current working directory.

Parameters:

bentofile – The file path to build config yaml file
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
bare – whether to build a bento without copying files
reload – whether to reload the service

Returns:

a Bento instance representing the materialized Bento saved in BentoStore

Return type:

Bento

Load models¶

class bentoml.models.BentoModel(tag: Tag | str)[source]¶

Bases: Model[Model]

A model reference to a BentoML model.

Parameters:: tag (Tag) – The model tag.
Returns:: The bento model object.
Return type:: Model

classmethod from_info(info: BentoModelInfo) → BentoModel[source]¶: Return the model object from the model info.

resolve(base_path: str | ~os.PathLike | ~fs.base.FS | None = None, global_model_store: ~bentoml._internal.models.model.ModelStore = <simple_di.providers.SingletonFactory object>) → Model[source]¶: Get the actual object of the model.

to_info(alias: str | None = None) → BentoModelInfo[source]¶: Return the model info object.

class bentoml.models.HuggingFaceModel(model_id: str, revision: str = 'main', endpoint: str | None = NOTHING)[source]¶

Bases: Model[str]

A model reference to a Hugging Face model.

Parameters:

model_id (str) – The model tag. E.g. “google-bert/bert-base-uncased”.
revision (str, optional) – The revision to use. Defaults to “main”.
endpoint (str, optional) – The Hugging Face endpoint to use. Defaults to https://huggingface.co.

Returns:

The downloaded model path.

Return type:

str

classmethod from_info(info: BentoModelInfo) → HuggingFaceModel[source]¶: Return the model object from the model info.

resolve(base_path: str | PathLike | FS | None = None) → str[source]¶: Get the actual object of the model.

to_info(alias: str | None = None) → BentoModelInfo[source]¶: Return the model info object.

Manage models¶

bentoml.models.create(name: Tag | str, *, labels: dict[str, t.Any] | None = None, metadata: dict[str, t.Any] | None = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, module: str = '', api_version: str = 'v1', signatures: ModelSignaturesType = {}, options: ModelOptions = ModelOptions(), custom_objects: dict[str, t.Any] | None = None, external_modules: t.List[ModuleType] | None = None, context: ModelContext = ModelContext(framework_name='', framework_versions={}, bentoml_version='1.3.15.post10+gca0a370c', python_version='3.11.10')) → t.Generator[Model, None, None][source]¶

Create a model instance in BentoML modelstore.

The following function arguments are deprecated:: module, api_version, signatures, options, custom_objects, external_modules, context

bentoml.models.list(tag: t.Optional[t.Union[Tag, str]] = None, *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → t.List['Model'][source]¶

bentoml.models.get(tag: t.Union[Tag, str], *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, model_aliases: t.Dict[str, str] = <simple_di.providers.Static object>) → Model[source]¶: Get a model by tag. If the tag is a string, it will be looked up in the model_aliases dict.

bentoml.models.delete(tag: t.Union[Tag, str], *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.export_model(tag: t.Union[Tag, str], path: str, output_format: t.Optional[str] = None, *, protocol: t.Optional[str] = None, user: t.Optional[str] = None, passwd: t.Optional[str] = None, params: t.Optional[t.Dict[str, str]] = None, subpath: t.Optional[str] = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → str[source]¶

Export a BentoML model.

Examples:

# exports 'my_model' to '/path/to/folder/my_model-version.bentomodel' in BentoML's default format
bentoml.models.export_model('my_model:latest', '/path/to/folder')
# note that folders can only be passed if exporting to the local filesystem; otherwise the
# full path, including the desired filename, must be passed

# exports 'my_model' to '/path/to/folder/my_model.bentomodel' in BentoML's default format
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model')
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.bentomodel')

# exports 'my_model' to '/path/to/folder/my_model.tar.gz in gzip format
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.tar.gz')
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.tar.gz', 'gz')

# exports 'my_model' to '/path/to/folder/my_model/ as a folder
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model', 'folder')

# exports 'my_model' to the S3 bucket 'my_bucket' as 'folder/my_model-version.bentomodel'
bentoml.models.export_model('my_model:latest', 's3://my_bucket/folder')
bentoml.models.export_model('my_model:latest', 'my_bucket/folder', protocol='s3')
bentoml.models.export_model('my_model:latest', 'my_bucket', protocol='s3', subpath='folder')
bentoml.models.export_model('my_model:latest', 'my_bucket', protocol='s3', subpath='folder',
                            user='<AWS access key>', passwd='<AWS secret key>',
                            params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the model to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL, for example, 's3://my_bucket/folder/my_model.bentomodel'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'.
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the model should be exported to
_model_store – the model store to get the model to save from

Returns:

A representation of the path that the model was exported to. If it was exported to the local filesystem,: this will be the OS path to the exported model. Otherwise, it will be an FS URL.

Return type:

str

bentoml.models.import_model(path: str, input_format: t.Optional[str] = None, *, protocol: t.Optional[str] = None, user: t.Optional[str] = None, passwd: t.Optional[str] = None, params: t.Optional[t.Dict[str, str]] = None, subpath: t.Optional[str] = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → Model[source]¶

Import a bento model exported with bentoml.models.export_model. To import a model saved with a framework, see the save function under the relevant framework, e.g. bentoml.sklearn.save.

Examples:

# imports 'my_model' from '/path/to/folder/my_model.bentomodel'
bentoml.models.import_model('/path/to/folder/my_model.bentomodel')

# imports 'my_model' from '/path/to/folder/my_model.tar.gz'
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.models.import_model('/path/to/folder/my_model.tar.gz')
# treats 'my_model.ext' as a gzipped tarfile
bentoml.models.import_model('/path/to/folder/my_model.ext', 'gz')

# imports 'my_model', which is stored as an uncompressed folder, from '/path/to/folder/my_model/'
bentoml.models.import_model('/path/to/folder/my_model', 'folder')

# imports 'my_model' from the S3 bucket 'my_bucket', path 'folder/my_model.bentomodel'
# requires `fs-s3fs <https://pypi.org/project/fs-s3fs/>`_ ('pip install fs-s3fs')
bentoml.models.import_model('s3://my_bucket/folder/my_model.bentomodel')
bentoml.models.import_model('my_bucket/folder/my_model.bentomodel', protocol='s3')
bentoml.models.import_model('my_bucket', protocol='s3', subpath='folder/my_model.bentomodel')
bentoml.models.import_model('my_bucket', protocol='s3', subpath='folder/my_model.bentomodel',
                            user='<AWS access key>', passwd='<AWS secret key>',
                            params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the model to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL, for example 's3://my_bucket/folder/my_model.bentomodel'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the model should be exported to
_model_store – the model store to save the model to

Returns:

the imported model

Return type:

Model

Work with BentoCloud¶

bentoml.push(tag: Tag | str, *, force: bool = False, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶: Push Bento to a yatai server.

bentoml.pull(tag: Tag | str, *, force: bool = False, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.push(tag: t.Union[Tag, str], *, force: bool = False, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.pull(tag: t.Union[Tag, str], *, force: bool = False, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>) → Model | None[source]¶