Sandboxes

Important

Sandbox is currently a preview feature. If you’d like early access, please reach out to us.

Modern AI applications like agentic systems often generate and execute code autonomously. However, letting these agents use tools, write scripts, and make decisions without human review is risky. Consider these common scenarios:

• An LLM agent writes Python code during its reasoning loop and runs it on the fly.

• A prompt injection leads the agent to execute harmful or unexpected logic.

• The agent integrates third-party tools or APIs with unpredictable behavior.

• The agent clones a GitHub repository and runs tasks like test suites and static analysis.

In each case, you can’t assume the code is safe. Untrusted code can damage your infrastructure or expose sensitive data.

A BentoML Sandbox provides an isolated, secure, and ephemeral environment for running untrusted or dynamically generated code. With Sandboxes, you can:

• Run code generated by LLMs in a contained environment.

• Customize each Sandbox environment with your choice of base image, packages, environment variables, and startup behavior.

• Scale workloads from 1 to thousands of instances with fast cold starts.

Create a Sandbox

You can create a Sandbox and specify its runtime:

# Use BentoCloud client
client = bentoml.BentoCloudClient()

sb = client.sandbox.create(
    image=bentoml.images.Image(python_version="3.12")
        .run("apt-get install -y curl")
        .python_packages("fastapi")
)

# Block until sandbox is ready (show logs if enabled)
sb.wait_until_ready(log_polling=True)

To run commands in a Sandbox:

result = sb.exec("python", "-c", "import sys;print(sys.version)")
print(result.stdout.read())

# Output streaming
result = sb.exec("python", "-c", "import sys;print(sys.version)")
for line in result.stdout:
    print(line)

On the Monitoring tab of the Sandbox details page, you can view real-time metrics such as replica count and resource usage:

Reuse an existing Sandbox

Previously created Sandboxes can be reused:

sb = bentoml.sandbox.get(name=sandbox_name)

# Using BentoCloud client
client = bentoml.BentoCloudClient()
sb = client.sandbox.get(name=sandbox_name)

Parameters

You can customize sandbox behavior via more parameters. For example:

# Custom command
sb = bentoml.sandbox.create(cmd=["python", "-m", "http.server"])

# Env vars and secrets
sb = bentoml.sandbox.create(secrets=["my-secret"], envs={"HF_TOKEN": "abdef"})

# Add labels
sb = bentoml.sandbox.create(labels={"foo": "bar"})

Full signature:

Parameter	Type	Description
image	Image | None	Base image specification (e.g. Python version, packages, OS dependencies).
labels	Mapping[str, str] | None	Optional key-value metadata to tag or group the sandbox.
cmd	list[str] | None	Custom command to run when the Sandbox starts.
secrets	list[str] | None	List of secret keys to mount into the Sandbox environment.
envs	Mapping[str, str] | None	Environment variables to set inside the Sandbox.
cluster	str | None	Optional BentoCloud cluster name to launch the Sandbox on.
client	BentoCloudClient	The BentoCloud client instance to use. It defaults to default_client.

Terminate a Sandbox

# Terminate and delete the sandbox
sb.destroy()

# Destroy a sandbox by name
bentoml.sandbox.destroy(name=sandbox_name)