TorchScript

About this page

This is the API reference for TorchScript in BentoML. You can find more information about TorchScript in the official documentation.

Note

You can find more examples for TorchScript in our bentoml/examples directory.

bentoml.torchscript.save_model(name: Tag | str, model: torch.ScriptModule, *, signatures: ModelSignaturesType | None = None, labels: t.Dict[str, str] | None = None, custom_objects: t.Dict[str, t.Any] | None = None, external_modules: t.List[ModuleType] | None = None, metadata: t.Dict[str, t.Any] | None = None, _framework_name: str = 'torchscript', _module_name: str = 'bentoml.torchscript', _extra_files: dict[str, t.Any] | None = None) → bentoml.Model

Save a model instance to BentoML modelstore.

Parameters:

• name (str) – Name for given model instance. This should pass Python identifier check.

• model (torch.ScriptModule) – Instance of model to be saved

• signatures (dict, optional) – A dictionary of method names and their corresponding signatures.

• labels (Dict[str, str], optional, default to None) – user-defined labels for managing models, e.g. team=nlp, stage=dev

• custom_objects (Dict[str, Any]], optional, default to None) – user-defined additional python objects to be saved alongside the model, e.g. a tokenizer instance, preprocessor function, model configuration json

• external_modules (List[ModuleType], optional, default to None) – user-defined additional python modules to be saved alongside the model or custom objects, e.g. a tokenizer module, preprocessor module, model configuration module

• metadata (Dict[str, Any], optional, default to None) – Custom metadata for given model.

Returns:

A tag with a format name:version where name is the user-defined model’s name, and a generated version by BentoML.

Return type:

Tag

Examples:

import bentoml
import torch

bentoml.torchscript.load_model(bentoml_model: str | Tag | Model, device_id: str | None = 'cpu', *, _extra_files: dict[str, t.Any] | None = None) → torch.ScriptModule | tuple[torch.ScriptModule, dict[str, t.Any]]

Load a model from BentoML local modelstore with given name.

Parameters:

• tag – Tag of a saved model in BentoML local modelstore.

• device_id – Optional devices to put the given model on. Refer to https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.device

• _extra_files – A dictionary of file names and a empty string. See https://pytorch.org/docs/stable/generated/torch.jit.load.html.

Returns:

an instance of torch.ScriptModule from BentoML modelstore.

Return type:

torch.ScriptModule

Examples:

import bentoml
lit = bentoml.torchscript.load_model('lit_classifier:latest', device_id="cuda:0")

bentoml.torchscript.get(tag_like: str | Tag) → Model