Isolation Sessions
Isolation sessions run a long-lived bash inside a bubblewrap namespace, so one sandbox pod can host many mutually isolated task runs without spinning up a new container. Each session gets its own PID, mount, tmpfs, and env namespaces; startup is sub-millisecond.
Table of Contents
- Requirements
- Overview
- How It Works
- Quick Start
- Session Lifecycle
- Workspace Modes
- Bind Mounts and Allowlist
- Profiles and Defaults
- Environment, UID, and Networking
- Filesystem Proxy
- Capabilities and Probing
- Server Configuration
- Limitations
- See Also
Requirements
Component versions needed for the features covered by this guide:
execd>= 1.0.20 for base isolation session support; >= 1.0.21 recommended forbinds,List sessions,uid_mode: "userns", and the default writable allowlist (/workspace,/mnt,/media,/data)opensandbox-server>= 0.2.1 — the server injectsCAP_SYS_ADMIN,apparmor=unconfined, and the tmpfs mount required bybwrapwhen the execd image declaresbootstrap.execd.isolation- Python SDK >= 0.1.14 (
isolation.run_once/isolation.sessioncontext manager); >= 0.1.13 for the generated isolation client only - JavaScript / TypeScript SDK >= 0.1.10 (
isolation.runOnce/isolation.withSession) - Kotlin / Java SDK >= 1.0.16 (
isolation.runOnce()/isolation.withSession { ... }); >= 1.0.15 for the generated isolation client only - C# SDK >= 0.1.4 (
RunOnceAsync/WithSessionAsync) - Go SDK >= 1.0.4 (
IsolationRunOnce/IsolationWithSession)
The setpriv_available / userns_available fields on /capabilities were added after execd v1.0.21 and are only present on execd builds from main at the time of writing; older execd omits both fields (clients must tolerate their absence).
Host requirements (bwrap binary, CAP_SYS_ADMIN, overlayfs, etc.) are listed under Server Configuration → Host Requirements.
The published execd image includes the fail-closed native workload gate used during session startup. For a Linux source build, make build requires a C compiler plus static libc and produces bin/opensandbox-session-gate; run make build-session-gate and then sudo make install-session-gate before starting execd to install the helper at /opt/opensandbox/opensandbox-session-gate. Keep that path and its parent directory root-owned and not group- or world-writable.
Overview
| Concept | Boundary | Startup | Typical Use |
|---|---|---|---|
| Isolation session | bubblewrap namespaces inside one sandbox | ~100 ms to create; subsequent run in an existing session is near-zero overhead | Many short, mutually isolated task runs reusing one long-lived session |
Bash session (/session) | bash process, no extra namespaces | ms | Interactive REPL-style command sequences |
| Sandbox | container or pod | 100s of ms to seconds | Tenant, workspace, or user boundary |
| Secure runtime (gVisor / Kata) | user-space kernel or VM per sandbox | 10–500 ms | Hardware-level protection against container escape |
Session creation vs. per-run cost
Creating a session takes about 100 ms because execd waits briefly after starting bwrap to detect an immediately-exiting child. Once the session exists, each POST /run reuses the same bash process, so per-run overhead is negligible. Design workloads that amortize the create cost across many run calls in one session.
Good fits: RL rollouts, batch code grading, multi-tool agent runs — one sandbox per worker, many isolated tasks inside.
Not a fit: cross-language kernels (use /code), interactive REPLs (use /session), or hard trust boundaries against kernel exploits (use gVisor / Kata; see Secure Container Runtime).
How It Works
execd forks one bwrap child per session; bwrap sets up Linux namespaces, then execs a long-lived bash inside them.
Two things the diagram doesn't show:
bashis long-lived, soexport X=1in onerunis visible to the nextrunin the same session (never in another session).- The FS proxy reads and writes the merged workspace view from outside the namespace, so uploads and downloads work while
bashis busy.
run request flow
Non-happy paths:
| Event | What happens |
|---|---|
Run timeout_seconds elapsed | execd cancels the run context, which sends SIGINT to the bwrap process group and emits an IsolatedError SSE event; the session itself stays alive. |
bwrap process exited | The next run returns an IsolatedError SSE event with session process has exited; only ErrContextNotFound (session ID unknown) becomes an HTTP-level error. DELETE + recreate. |
| Idle timeout reached | GC runs the same teardown as DELETE. |
| Client disconnects mid-SSE | The Gin request context is cancelled and execd sends SIGINT to the running command; the run does not continue in the background. |
Quick Start
curl
# Probe.
curl -s http://localhost:44772/v1/isolated/capabilities
# Create.
SESSION=$(curl -s -X POST http://localhost:44772/v1/isolated/session \
-H "Content-Type: application/json" \
-d '{
"profile": "strict",
"workspace": {"path": "/workspace", "mode": "overlay"},
"idle_timeout_seconds": 300
}' | jq -r .session_id)
# Run (SSE: stdout / error / complete).
curl -N -X POST "http://localhost:44772/v1/isolated/session/$SESSION/run" \
-H "Content-Type: application/json" \
-d '{"code": "export X=1; echo $X", "timeout_seconds": 30}'
# Second run reuses shell state.
curl -N -X POST "http://localhost:44772/v1/isolated/session/$SESSION/run" \
-H "Content-Type: application/json" \
-d '{"code": "echo $X"}' # prints 1
# Delete.
curl -X DELETE "http://localhost:44772/v1/isolated/session/$SESSION"Python SDK
from opensandbox import Sandbox
from opensandbox.models.isolated import (
CreateIsolatedSessionRequest, IsolatedWorkspaceSpec, IsolatedRunOpts,
)
# Sandbox.create is an async classmethod, so await it before entering
# the async context manager.
async with (await Sandbox.create("python:3.11")) as sandbox:
# One-shot.
await sandbox.isolation.run_once(
"python -c 'print(42)'", workspace="/workspace", profile="strict",
)
# Persistent session.
async with sandbox.isolation.session(
CreateIsolatedSessionRequest(
workspace=IsolatedWorkspaceSpec(path="/workspace", mode="overlay"),
profile="strict",
idle_timeout_seconds=300,
)
) as session:
await session.run("export STAGE=train")
await session.run("python train.py", opts=IsolatedRunOpts(timeout_seconds=600))
await session.files.write_file("/workspace/data.csv", data_bytes)
# Reattach after a client restart.
handle = await sandbox.isolation.attach(known_session_id)JavaScript / TypeScript SDK
await sandbox.isolation.runOnce("node -e 'console.log(1)'", "/workspace", {
profile: "strict",
});
await sandbox.isolation.withSession(
{ profile: "strict", workspace: { path: "/workspace", mode: "overlay" } },
async (session) => { await session.run("npm test"); },
);Kotlin SDK
sandbox.isolation().runOnce(code = "python -c 'print(1)'", workspace = "/workspace")
sandbox.isolation().withSession(request) { session ->
session.run("ls /workspace")
}Session Lifecycle
Under /v1/isolated/ on execd. Use X-EXECD-ACCESS-TOKEN when execd has an access token configured.
| Method | Path | Purpose |
|---|---|---|
POST | /session | Create; returns {session_id, created_at}. |
GET | /sessions | List active sessions. |
GET | /session/{id} | Full state; echoes creation params so a stateless client can rebuild the handle. |
POST | /session/{id}/run | Foreground (default): SSE stream stdout / error / complete. Background ("background": true): returns 202 with a run handle. Runs on the same session are serialized. |
GET | /session/{id}/runs/{runId} | Background run status: running, exit_code, error, timestamps. |
GET | /session/{id}/runs/{runId}/logs | Background run combined output as plain text, with byte-cursor pagination. |
DELETE | /session/{id} | Destroy. |
GET | /capabilities | Probe. |
idle_timeout_seconds > 0 destroys idle sessions automatically; set to 0 to disable idle GC and always DELETE explicitly.
Background Runs
A background run starts code detached inside the session and returns immediately; the run's combined stdout/stderr and exit code are captured by execd, and the client polls them while other work continues on the session.
# Start detached; 202 returns {"session_id", "run_id", "started_at"}.
RUN=$(curl -s -X POST "http://localhost:44772/v1/isolated/session/$SESSION/run" \
-H "Content-Type: application/json" \
-d '{"code": "sleep 5 && echo done", "background": true}' | jq -r .run_id)
# Poll status until running=false.
curl -s "http://localhost:44772/v1/isolated/session/$SESSION/runs/$RUN"
# Read output (plain text). The response header EXECD-ISOLATED-TAIL-CURSOR
# carries the next byte offset for incremental reads.
curl -s "http://localhost:44772/v1/isolated/session/$SESSION/runs/$RUN/logs"
curl -s "http://localhost:44772/v1/isolated/session/$SESSION/runs/$RUN/logs?cursor=123"Python SDK:
session = await sandbox.isolation.create(CreateIsolatedSessionRequest(
workspace=IsolatedWorkspaceSpec(path="/workspace", mode="rw"),
))
run = await session.run_background("sleep 5 && echo done")
while (status := await session.run_status(run.run_id)).running:
await asyncio.sleep(0.2)
print(status.exit_code)
logs = await session.run_logs(run.run_id) # .text + .cursor for paginationBackground run semantics:
timeout_secondsis foreground-only; background runs are not time-limited.- Idle GC is suspended while a background run is active; after it finishes, the normal idle window applies. Deleting the session kills the run.
- Background runs require a writable log location, so sessions with a read-only (
ro) workspace reject them with400;rwandoverlayworkspaces are supported. - Runs share the session's process group, so session-level signals (e.g. the
SIGINTsent when a foreground run times out or is cancelled) also reach them. - Each
logsrequest returns at most 16 MiB; page through large output with the returned cursor. Per-run log retention is capped at 16 MiB — output beyond the cap is discarded when the run completes, so drain incrementally while the run is active if you need more than the first page. - A run whose session dies mid-flight reports
running: falsewitherror: "session terminated"; run records and their logs are removed when the session is deleted or garbage-collected.
Workspace Modes
| Mode | Semantics |
|---|---|
rw | Bind-mount read-write; writes persist on the host. |
overlay (default) | Copy-on-write via overlayfs; writes go to a per-session upper dir and vanish on DELETE. |
ro | Bind-mount read-only; writes fail with EROFS. |
Overlay upper dirs live under upper_root (default /var/lib/execd/isolation).
{ "workspace": { "path": "/workspace", "mode": "overlay" } }Bind Mounts and Allowlist
extra_writable— paths bind-mounted read-write at the same path (source == destination).binds— explicitsource→destmappings, optionallyreadonly.sourcemust already exist;destmust already exist inside the namespace (bake it into the sandbox image).
{
"workspace": { "path": "/workspace", "mode": "rw" },
"extra_writable": ["/data/scratch"],
"binds": [
{ "source": "/data/in", "dest": "/mnt/in", "readonly": true },
{ "source": "/data/out", "dest": "/mnt/out" }
]
}Every source is checked against the operator-configured allowed_writable allowlist after symlink resolution, so symlinks cannot escape it. Default allowlist: /workspace, /mnt, /media, /data (subpaths allowed). Empty allowlist rejects all extra binds.
Profiles and Defaults
The profile field currently controls only how /tmp is exposed inside the namespace:
| Profile | /tmp |
|---|---|
strict (default) | Private tmpfs (--tmpfs /tmp) |
balanced | Shared with the sandbox (--bind /tmp /tmp) |
workspace.mode and env_passthrough are independent of the profile: when they are omitted, execd normalizes workspace.mode to overlay and env_passthrough.mode to deny regardless of which profile you pick. Set those fields explicitly if you want persistent workspace writes ("rw") or host env passthrough ("allow").
Environment, UID, and Networking
env_passthrough—mode: "allow"+keyswhitelists host env vars; defaultdeny. Per-run overrides go inIsolatedRunRequest.envs.uid/gidwithuid_mode: "setpriv"(default, real setuid/setgid drop) or"userns"(user namespace remap). Checksetpriv_available/userns_availablefrom/capabilitiesbefore requesting a mode.share_net: trueshares the sandbox's network namespace. Sandbox-level egress and Credential Vault policies still apply.share_net: falsecreates a private network namespace. Before releasing the workload startup gate, execd opens the authenticated NetNS, obtains its real owning UserNS withNS_GET_USERNS, and bind-pins both below/run/execd/namespaces/<opaque-id>/. Any validation or pin failure aborts Session creation. Execd attempts pin cleanup on failed startup, process exit, explicit delete, idle collection, and runner shutdown; retryable failures retain Session ownership and are retried while execd remains alive. This applies to both UID modes; hardened network backends will requireuid_mode: "userns".
The legacy default remains unchanged in this phase: omitting share_net continues to share the sandbox network namespace. Namespace pinning alone does not enable Session egress or ingress.
Private Sessions are not recoverable across an execd restart. Deployments that enable them must treat execd as sandbox-critical and recreate the entire sandbox/container if execd exits; they must not launch a replacement execd inside the surviving mount namespace. Destroying the sandbox/container tears down that mount namespace and releases all pins. Execd intentionally does not scan and adopt opaque namespace-pin directories on startup because they do not carry a durable sandbox generation, so doing so could unmount another live process's resources.
Filesystem Proxy
Reads and writes the session's merged workspace view from outside the namespace — this is how SDKs upload, download, and list without spawning a shell. All paths are per session under /v1/isolated/session/{id}/:
| Method | Path |
|---|---|
GET | files/info?path=... |
GET | files/download?path=... (supports Range and offset/limit) |
POST | files/upload (multipart: metadata + file) |
DELETE | files?path=... |
POST | files/mv |
POST | files/permissions |
POST | files/replace?verbose=true |
GET | files/search?path=...&pattern=... |
GET | directories/list?path=...&depth=N |
POST | directories |
DELETE | directories?path=... |
Writes on a ro workspace fail; writes on overlay land in the upper dir and vanish on DELETE.
Capabilities and Probing
curl -s http://localhost:44772/v1/isolated/capabilities{
"available": true,
"isolator": "bwrap",
"version": "0.9.0",
"setpriv_available": true,
"userns_available": false,
"commit_supported": false,
"diff_supported": false
}available: false— the trusted native workload gate is missing or untrusted, bubblewrap is missing, or the host can't create the required namespaces (missingCAP_SYS_ADMIN, restricted user-ns sysctl, etc.).setpriv_available/userns_available— whether sessions withuid_mode: "setpriv"or"userns"can be created.setpriv_availablereflects only execd's default UID/GID; a session that requests a different UID/GID may still return503 NOT_SUPPORTEDwhen identity switching is unavailable.commit_supported/diff_supported— Phase 2 stubs, currently return503.
Server Configuration
Point execd at an optional TOML file:
| Flag | Env |
|---|---|
--isolation-config | EXECD_ISOLATION_CONFIG |
# Parent directory for per-session overlay upper dirs.
upper_root = "/var/lib/execd/isolation"
# Hard limit on total upper directory size across all sessions (bytes).
# Default: 8 GiB. Set to 0 only if you want to disable the quota entirely.
upper_max_bytes = 8589934592 # 8 GiB
# Sources allowed for extra_writable / binds (symlink-resolved).
# Default: ["/workspace", "/mnt", "/media", "/data"]. Empty = reject all.
allowed_writable = ["/workspace", "/mnt", "/media", "/data"]Example: components/execd/configs/isolation.example.toml.
Host requirements: bwrap and the trusted native workload gate in the execd image; CAP_SYS_ADMIN (and kernel.unprivileged_userns_clone=1 for uid_mode: "userns"); overlayfs in the kernel for overlay workspaces. The published image installs the gate automatically. Linux source builds must run make build-session-gate and then sudo make install-session-gate from components/execd before starting execd.
Note: /capabilities reports available: false when the native workload gate cannot be opened as a trusted executable or when bwrap itself cannot be started (missing binary or missing namespace capabilities). A missing overlayfs does not flip available — the overlay probe only influences Phase 2 commit/diff support, and default overlay-mode session creation can still fail at runtime on such hosts. If you rely on workspace.mode: "overlay", verify overlayfs support directly on the host.
Limitations
diff/commitare Phase 2 stubs, currently return503. Tracked in OSEP-0013.- No hardware-level guarantee. Namespaces + seccomp only; pair with a secure runtime for kernel-exploit defense.
- Linux only. Non-Linux builds return
available: false. - Serialized runs per session — create multiple sessions for parallelism.
- Bind destinations must pre-exist in the sandbox image.