MLflow¶

MLflow is an open-source platform, purpose-built to assist machine learning practitioners and teams in handling the complexities of the machine learning process. MLflow focuses on the full lifecycle for machine learning projects, ensuring that each phase is manageable, traceable, and reproducible.

This document explains how to serve and deploy an MLflow model with BentoML.

Source Code

Deploy to BentoCloud

Serve with BentoML

The example uses scikit-learn for demo purposes. You can submit a classification request to the endpoint like this:

{
     "input_data": [[5.9,3,5.1,1.8]]
}

Expected output:

["virginica"]

In addition to scikit-learn, both MLflow and BentoML support a wide variety of other frameworks, such as PyTorch, TensorFlow and XGBoost.

This example is ready for quick deployment and scaling on BentoCloud. With a single command, you get a production-grade application with fast autoscaling, secure deployment in your cloud, and comprehensive observability.

Screenshot of MLflow Iris classifier model deployed on BentoCloud showing the prediction interface with input data and classification results

Code explanations¶

You can find the source code in GitHub. Below is a breakdown of the key code implementations within this project.

save_model.py¶

This example uses the scikit-learn framework to train a classification model and saves it with MLflow.

save_model.py¶

from sklearn.datasets import load_iris
from sklearn.neighbors import KNeighborsClassifier
from pathlib import Path
import mlflow.sklearn

iris = load_iris()
X_train = iris.data[:, :4]
Y_train = iris.target

model_uri = Path("models", "IrisClf")
model = KNeighborsClassifier()
model.fit(X_train, Y_train)
mlflow.sklearn.save_model(model, model_uri.resolve())

After training, use the bentoml.mlflow.import_model API to save the model to the BentoML Model Store, a local directory to store and manage models. You can retrieve this model later in other services to run predictions.

import bentoml

# model_uri can be any URI that refers to an MLflow model
# Use local path for demonstration
bentoml.mlflow.import_model("iris", model_uri)

To verify that the model has been successfully saved, run:

$ bentoml models list

Tag                      Module           Size       Creation Time
iris:74px7hboeo25fjjt    bentoml.mlflow   10.07 KiB  2024-06-19 10:09:21

test.py¶

To ensure that the saved model works correctly, try loading it and running a prediction:

test.py¶

import numpy as np
import bentoml

# Load the latest version of iris model
iris_model = bentoml.mlflow.load_model("iris:latest")

input_data = np.array([[5.9, 3, 5.1, 1.8]])
res = iris_model.predict(input_data)
print(res)

Expected result:

[2] # The model thinks the category seems to be Virginica.

service.py¶

The service.py file is where you define the serving logic and expose the model as a web service.

service.py¶

import bentoml
import numpy as np

target_names = ['setosa', 'versicolor', 'virginica']

@bentoml.service(
    resources={"cpu": "2"},
    traffic={"timeout": 10},
)
class IrisClassifier:
    # Declare the model as a class variable
    bento_model = bentoml.models.BentoModel("iris:latest")

    def __init__(self):
        self.model = bentoml.mlflow.load_model(self.bento_model)

    @bentoml.api
    def predict(self, input_data: np.ndarray) -> list[str]:
        preds = self.model.predict(input_data)
        return [target_names[i] for i in preds]

The Service code:

Uses the @bentoml.service decorator to define a BentoML Service. Optionally, you can set additional configurations like resource allocation on BentoCloud and traffic timeout.
Retrieves the model from the Model Store and defines it a class variable.
Uses the @bentoml.api decorator to expose the predict function as an API endpoint.

The @bentoml.service decorator also allows you to define the runtime environment for a Bento, the unified distribution format in BentoML. A Bento is packaged with all the source code, Python dependencies, model references, and environment setup, making it easy to deploy consistently across different environments.

Here is an example:

service.py¶

my_image = bentoml.images.Image(python_version="3.11") \
            .python_packages("mlflow", "scikit-learn")

@bentoml.service(
    image=my_image, # Apply the specifications
    ...
)
class IrisClassifier:
    ...

Try it out¶

You can run this example project on BentoCloud, or serve it locally, containerize it as an OCI-compliant image and deploy it anywhere.

BentoCloud¶

BentoCloud provides fast and scalable infrastructure for building and scaling AI applications with BentoML in the cloud.

Install the dependencies and log in to BentoCloud through the BentoML CLI. If you don’t have a BentoCloud account, sign up here for free.
```
# Recommend Python 3.11
pip install bentoml mlflow scikit-learn

bentoml cloud login
```

Clone the repository.

git clone https://github.com/bentoml/BentoMLflow.git
cd BentoMLflow

Train and save the MLflow model to the BentoML Model Store.
```
python3 save_model.py
```

Deploy the Service to BentoCloud.

bentoml deploy service.py:IrisClassifier

Once it is up and running, you can call the endpoint in the following ways:

BentoCloud Playground

Python client

Create a BentoML client to call the endpoint. Make sure you replace the Deployment URL with your own on BentoCloud. Refer to Obtain the endpoint URL for details.

import bentoml

with bentoml.SyncHTTPClient("https://iris-classifier-bdbe-e3c1c7db.mt-guc1.bentoml.ai") as client:
      result = client.predict(
          input_data=[
              [5.9, 3, 5.1, 1.8]
          ],
      )
      print(result)

CURL

Make sure you replace the Deployment URL with your own on BentoCloud. Refer to Obtain the endpoint URL for details.

curl -X 'POST' \
      'https://iris-classifier-bdbe-e3c1c7db.mt-guc1.bentoml.ai/predict' \
      -H 'accept: application/json' \
      -H 'Content-Type: application/json' \
      -d '{
      "input_data": [
          [5.9, 3, 5.1, 1.8]
      ]
  }'

To make sure the Deployment automatically scales within a certain replica range, add the scaling flags:
```
bentoml deploy --scaling-min 0 --scaling-max 3 # Set your desired count
```
If it’s already deployed, update its allowed replicas as follows:
```
bentoml deployment update <deployment-name> --scaling-min 0 --scaling-max 3 # Set your desired count
```
For more information, see how to configure concurrency and autoscaling.

Local serving¶

BentoML allows you to run and test your code locally, so that you can quickly validate your code with local compute resources.

Clone the project repository and install the dependencies.

git clone https://github.com/bentoml/BentoMLflow.git
cd BentoMLflow

# Recommend Python 3.11
pip install bentoml mlflow scikit-learn

Train and save the model to the BentoML Model Store.
```
python3 save_model.py
```

Serve it locally.

bentoml serve service.py:IrisClassifier

Visit or send API requests to http://localhost:3000.

For custom deployment in your own infrastructure, use BentoML to generate an OCI-compliant image.