Bento and model APIs¶

This page contains management APIs for Bentos and models.

Manage Bentos¶

bentoml.list(tag: Tag | str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → t.List[Bento][source]¶

bentoml.get(tag: Tag | str, *, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]¶

bentoml.delete(tag: Tag | str, *, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>)[source]¶

Export a bento.

To export a bento to S3, you must install BentoML with extras aws:

» pip install bentoml[aws]

Examples:

# exports 'my_bento' to '/path/to/folder/my_bento-version.bento' in BentoML's default format
bentoml.export_bento('my_bento:latest', '/path/to/folder')
# note that folders can only be passed if exporting to the local filesystem; otherwise the
# full path, including the desired filename, must be passed

# exports 'my_bento' to '/path/to/folder/my_bento.bento' in BentoML's default format
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento')
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.bento')

# exports 'my_bento' to '/path/to/folder/my_bento.tar.gz' in gzip format
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.tar.gz')
# outputs a gzipped tarfile as 'my_bento.ext'
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento.ext', 'gz')

# exports 'my_bento' to '/path/to/folder/my_bento/' as a folder
bentoml.export_bento('my_bento:latest', '/path/to/folder/my_bento', 'folder')

# exports 'my_bento' to the S3 bucket 'my_bucket' as 'folder/my_bento-version.bento'
bentoml.export_bento('my_bento:latest', 's3://my_bucket/folder')
bentoml.export_bento('my_bento:latest', 'my_bucket/folder', protocol='s3')
bentoml.export_bento('my_bento:latest', 'my_bucket', protocol='s3', subpath='folder')
bentoml.export_bento('my_bento:latest', 'my_bucket', protocol='s3', subpath='folder',
                     user='<AWS access key>', passwd='<AWS secret key>',
                     params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the Bento to export
path – can be either: * a folder on the local filesystem * an FS URL. For example, 's3://my_bucket/folder/my_bento.bento'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the bento should be exported to
_bento_store – save Bento created to this BentoStore

Returns:

A representation of the path that the Bento was exported to. If it was exported to the local filesystem,: this will be the OS path to the exported Bento. Otherwise, it will be an FS URL.

Return type:

str

bentoml.import_bento(path: str, input_format: str | None = None, *, protocol: str | None = None, user: str | None = None, passwd: str | None = None, params: t.Optional[t.Dict[str, str]] = None, subpath: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]¶

Import a bento.

Examples:

# imports 'my_bento' from '/path/to/folder/my_bento.bento'
bentoml.import_bento('/path/to/folder/my_bento.bento')

# imports 'my_bento' from '/path/to/folder/my_bento.tar.gz'
# currently supported formats are tar.gz ('gz'),
# tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.import_bento('/path/to/folder/my_bento.tar.gz')
# treats 'my_bento.ext' as a gzipped tarfile
bentoml.import_bento('/path/to/folder/my_bento.ext', 'gz')

# imports 'my_bento', which is stored as an
# uncompressed folder, from '/path/to/folder/my_bento/'
bentoml.import_bento('/path/to/folder/my_bento', 'folder')

# imports 'my_bento' from the S3 bucket 'my_bucket',
# path 'folder/my_bento.bento'
# requires `fs-s3fs <https://pypi.org/project/fs-s3fs/>`_
bentoml.import_bento('s3://my_bucket/folder/my_bento.bento')
bentoml.import_bento('my_bucket/folder/my_bento.bento', protocol='s3')
bentoml.import_bento('my_bucket', protocol='s3',
                     subpath='folder/my_bento.bento')
bentoml.import_bento('my_bucket', protocol='s3',
                     subpath='folder/my_bento.bento',
                     user='<AWS access key>', passwd='<AWS secret key>',
                     params={'acl': 'public-read',
                             'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd,

params, and subpath) mean, see the

FS URL documentation.

Parameters:

tag – the tag of the bento to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL,

for example 's3://my_bucket/folder/my_bento.bento'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the bento should be exported to
_bento_store – the bento store to save the bento to

Returns:

the imported bento

Return type:

Bento

Manage models¶

bentoml.models.create(name: Tag | str, *, labels: dict[str, t.Any] | None = None, metadata: dict[str, t.Any] | None = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, module: str = '', api_version: str = 'v1', signatures: ModelSignaturesType = {}, options: ModelOptions = ModelOptions(), custom_objects: dict[str, t.Any] | None = None, external_modules: t.List[ModuleType] | None = None, context: ModelContext = ModelContext(framework_name='', framework_versions={}, bentoml_version='1.3.7.post5+g6c610745', python_version='3.11.9')) → t.Generator[Model, None, None][source]¶

Create a model instance in BentoML modelstore.

The following function arguments are deprecated:: module, api_version, signatures, options, custom_objects, external_modules, context

bentoml.models.list(tag: t.Optional[t.Union[Tag, str]] = None, *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → t.List['Model'][source]¶

bentoml.models.get(tag: t.Union[Tag, str], *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, model_aliases: t.Dict[str, str] = <simple_di.providers.Static object>) → Model[source]¶: Get a model by tag. If the tag is a string, it will be looked up in the model_aliases dict.

bentoml.models.delete(tag: t.Union[Tag, str], *, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.export_model(tag: t.Union[Tag, str], path: str, output_format: t.Optional[str] = None, *, protocol: t.Optional[str] = None, user: t.Optional[str] = None, passwd: t.Optional[str] = None, params: t.Optional[t.Dict[str, str]] = None, subpath: t.Optional[str] = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → str[source]¶

Export a BentoML model.

Examples:

# exports 'my_model' to '/path/to/folder/my_model-version.bentomodel' in BentoML's default format
bentoml.models.export_model('my_model:latest', '/path/to/folder')
# note that folders can only be passed if exporting to the local filesystem; otherwise the
# full path, including the desired filename, must be passed

# exports 'my_model' to '/path/to/folder/my_model.bentomodel' in BentoML's default format
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model')
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.bentomodel')

# exports 'my_model' to '/path/to/folder/my_model.tar.gz in gzip format
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.tar.gz')
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model.tar.gz', 'gz')

# exports 'my_model' to '/path/to/folder/my_model/ as a folder
bentoml.models.export_model('my_model:latest', '/path/to/folder/my_model', 'folder')

# exports 'my_model' to the S3 bucket 'my_bucket' as 'folder/my_model-version.bentomodel'
bentoml.models.export_model('my_model:latest', 's3://my_bucket/folder')
bentoml.models.export_model('my_model:latest', 'my_bucket/folder', protocol='s3')
bentoml.models.export_model('my_model:latest', 'my_bucket', protocol='s3', subpath='folder')
bentoml.models.export_model('my_model:latest', 'my_bucket', protocol='s3', subpath='folder',
                            user='<AWS access key>', passwd='<AWS secret key>',
                            params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the model to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL, for example, 's3://my_bucket/folder/my_model.bentomodel'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'.
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the model should be exported to
_model_store – the model store to get the model to save from

Returns:

A representation of the path that the model was exported to. If it was exported to the local filesystem,: this will be the OS path to the exported model. Otherwise, it will be an FS URL.

Return type:

str

bentoml.models.import_model(path: str, input_format: t.Optional[str] = None, *, protocol: t.Optional[str] = None, user: t.Optional[str] = None, passwd: t.Optional[str] = None, params: t.Optional[t.Dict[str, str]] = None, subpath: t.Optional[str] = None, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) → Model[source]¶

Import a bento model exported with bentoml.models.export_model. To import a model saved with a framework, see the save function under the relevant framework, e.g. bentoml.sklearn.save.

Examples:

# imports 'my_model' from '/path/to/folder/my_model.bentomodel'
bentoml.models.import_model('/path/to/folder/my_model.bentomodel')

# imports 'my_model' from '/path/to/folder/my_model.tar.gz'
# currently supported formats are tar.gz ('gz'), tar.xz ('xz'), tar.bz2 ('bz2'), and zip
bentoml.models.import_model('/path/to/folder/my_model.tar.gz')
# treats 'my_model.ext' as a gzipped tarfile
bentoml.models.import_model('/path/to/folder/my_model.ext', 'gz')

# imports 'my_model', which is stored as an uncompressed folder, from '/path/to/folder/my_model/'
bentoml.models.import_model('/path/to/folder/my_model', 'folder')

# imports 'my_model' from the S3 bucket 'my_bucket', path 'folder/my_model.bentomodel'
# requires `fs-s3fs <https://pypi.org/project/fs-s3fs/>`_ ('pip install fs-s3fs')
bentoml.models.import_model('s3://my_bucket/folder/my_model.bentomodel')
bentoml.models.import_model('my_bucket/folder/my_model.bentomodel', protocol='s3')
bentoml.models.import_model('my_bucket', protocol='s3', subpath='folder/my_model.bentomodel')
bentoml.models.import_model('my_bucket', protocol='s3', subpath='folder/my_model.bentomodel',
                            user='<AWS access key>', passwd='<AWS secret key>',
                            params={'acl': 'public-read', 'cache-control': 'max-age=2592000,public'})

For a more comprehensive description of what each of the keyword arguments (protocol, user, passwd, params, and subpath) mean, see the FS URL documentation.

Parameters:

tag – the tag of the model to export
path –
can be one of two things: * a folder on the local filesystem * an FS URL, for example 's3://my_bucket/folder/my_model.bentomodel'
protocol – (expert) The FS protocol to use when exporting. Some example protocols are 'ftp', 's3', and 'userdata'
user – (expert) the username used for authentication if required, e.g. for FTP
passwd – (expert) the username used for authentication if required, e.g. for FTP
params – (expert) a map of parameters to be passed to the FS used for export, e.g. {'proxy': 'myproxy.net'} for setting a proxy for FTP
subpath – (expert) the path inside the FS that the model should be exported to
_model_store – the model store to save the model to

Returns:

the imported model

Return type:

Model

Work with BentoCloud¶

bentoml.push(tag: Tag | str, *, force: bool = False, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶: Push Bento to a yatai server.

bentoml.pull(tag: Tag | str, *, force: bool = False, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.push(tag: t.Union[Tag, str], *, force: bool = False, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>)[source]¶

bentoml.models.pull(tag: t.Union[Tag, str], *, force: bool = False, _cloud_client: BentoCloudClient = <simple_di.providers.SingletonFactory object>) → Model | None[source]¶