Client API

You can create a BentoML client to interact with your BentoML Service.

bentoml.SyncHTTPClient

class bentoml.SyncHTTPClient(url: str, *, token: str | None = None, timeout: float = 30, server_ready_timeout: float | None = None)

A synchronous client for BentoML service.

Parameters:

• url (str) – URL of the BentoML service.

• token (str, optional) – Authentication token. Defaults to None.

• timeout (float, optional) – Timeout for the client. Defaults to 30.

• server_ready_timeout (float, optional) – Timeout for the server to be ready. If not provided, it will use the same value as timeout.

Example:

with SyncHTTPClient("http://localhost:3000") as client:
    resp = client.call("classify", input_series=[[1,2,3,4]])
    assert resp == [0]
    # Or using named method directly
    resp = client.classify(input_series=[[1,2,3,4]])
    assert resp == [0]

client_cls

alias of Client

is_ready(timeout: int | None = None) → bool

bentoml.AsyncHTTPClient

class bentoml.AsyncHTTPClient(url: str, *, token: str | None = None, timeout: float = 30, server_ready_timeout: float | None = None)

An asynchronous client for BentoML service.

Parameters:

• url (str) – URL of the BentoML service.

• token (str, optional) – Authentication token. Defaults to None.

• timeout (float, optional) – Timeout for the client. Defaults to 30.

• server_ready_timeout (float, optional) – Timeout for the server to be ready. If not provided, it will use the same value as timeout.

Example:

async with AsyncHTTPClient("http://localhost:3000") as client:
    resp = await client.call("classify", input_series=[[1,2,3,4]])
    assert resp == [0]
    # Or using named method directly
    resp = await client.classify(input_series=[[1,2,3,4]])
    assert resp == [0]

    # Streaming
    resp = client.stream(prompt="hello")
    async for data in resp:
        print(data)

client_cls

alias of AsyncClient

async is_ready(timeout: int | None = None) → bool