slen.ai
Get started

Use the SDK in a FastAPI or a Django application

Python

One client for each worker process, a copy of it that logs with the id of each request, runs handed to Slen without waiting in a handler, and a client that never crosses a fork.

6 min read

This article is about the Python SDK.

The Python SDK has no integration of its own for a framework: a FastAPI or a Django application builds the client and calls it like any other code. What a server needs is the place the client is built in, a copy of it for each request, and a handler that does not wait for a run. This page shows both frameworks; the tests of the SDK run their code in the test client of each.

One client for each process

A server runs in several worker processes, and each serves many requests. Build one client in each process, and serve every request of that process with it.

  • A client keeps its connections and its settings, never the state of a call, so one client serves every request of its process — its threads and its tasks at once.

  • A client is built in the process that uses it, never when a module is imported: a client made before a server forks its workers would share its connections between them — below.

  • AsyncSlen belongs to the event loop it was built in, so an asyncio server builds it inside that loop.

FastAPI

Python
import logging
from collections.abc import AsyncGenerator, MutableMapping
from contextlib import asynccontextmanager
from typing import Any

from fastapi import FastAPI, Request

from slen import AsyncSlen


# One client per worker process, built in the worker's own event loop when it starts and closed when it stops —
# never when the module is imported, so workers forked from one process share no connections.
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
    async with AsyncSlen() as slen:
        app.state.slen = slen
        yield


app = FastAPI(lifespan=lifespan)


# Keeps the fields of each event of the SDK and adds the id of the request being served.
class RequestLog(logging.LoggerAdapter[logging.Logger]):
    def process(self, msg: Any, kwargs: MutableMapping[str, Any]) -> tuple[Any, MutableMapping[str, Any]]:
        return msg, {**kwargs, "extra": {**(kwargs.get("extra") or {}), **(self.extra or {})}}


def slen_of(request: Request) -> AsyncSlen:
    # A copy of the worker's client that logs with this request's id; the shared client does not change.
    shared: AsyncSlen = request.app.state.slen
    log = RequestLog(logging.getLogger("orders"), {"http_request_id": request.headers.get("x-request-id", "")})

    return shared.with_logger(log)

  • The client is built in the lifespan of the application, which each worker process runs in its own event loop when it starts, and async with closes it when the worker stops.

  • slen_of() gives each request a copy of the client — A log for each request.

Python
@app.post("/orders/{order_id}/email", status_code=202)
async def email_the_order(order_id: int, request: Request) -> dict[str, str]:
    # The run goes to a worker and the request is answered at once: a handler does not wait for a run.
    run = await slen_of(request).operations.execute_async(
        "generate_email",
        {"messages": [{"role": "user", "content": f"Write a short email: order {order_id} has shipped."}]},
        idempotency_key=f"order-{order_id}:email",
    )

    return {"execution_id": run.execution_id, "status": run.status}
Python
@app.get("/runs/{execution_id}")
async def run_status(execution_id: str, request: Request) -> dict[str, str]:
    # One read of the run; its end arrives as the execution.finished webhook.
    run = await slen_of(request).executions.get(execution_id)

    return {"execution_id": run.execution_id, "status": run.status}

Django

Python
import functools
import logging
from collections.abc import MutableMapping
from typing import Any

from django.http import HttpRequest, JsonResponse

from slen import Slen


# One client per worker process, built by the first request the process serves — never when the module is
# imported or the app is ready, so workers forked after loading the application share no connections.
@functools.cache
def shared_slen() -> Slen:
    return Slen()


# Keeps the fields of each event of the SDK and adds the id of the request being served.
class RequestLog(logging.LoggerAdapter[logging.Logger]):
    def process(self, msg: Any, kwargs: MutableMapping[str, Any]) -> tuple[Any, MutableMapping[str, Any]]:
        return msg, {**kwargs, "extra": {**(kwargs.get("extra") or {}), **(self.extra or {})}}


def slen_of(request: HttpRequest) -> Slen:
    # A copy of the process's client that logs with this request's id; the shared client does not change.
    log = RequestLog(logging.getLogger("orders"), {"http_request_id": request.headers.get("X-Request-Id", "")})

    return shared_slen().with_logger(log)

  • shared_slen() builds the client on the first request a process serves, and functools.cache keeps it for the rest — not when the module is imported, and not when the application is ready, which a server that loads the application before it forks its workers does before the fork.

  • A view of Django is synchronous, so it calls Slen.

Python
def email_the_order(request: HttpRequest, order_id: int) -> JsonResponse:
    # The run goes to a worker and the request is answered at once: a view does not wait for a run.
    run = slen_of(request).operations.execute_async(
        "generate_email",
        {"messages": [{"role": "user", "content": f"Write a short email: order {order_id} has shipped."}]},
        idempotency_key=f"order-{order_id}:email",
    )

    return JsonResponse({"execution_id": run.execution_id, "status": run.status}, status=202)
Python
def run_status(request: HttpRequest, execution_id: str) -> JsonResponse:
    # One read of the run; its end arrives as the execution.finished webhook.
    run = slen_of(request).executions.get(execution_id)

    return JsonResponse({"execution_id": run.execution_id, "status": run.status})
Python
from django.urls import path
from django.views.decorators.http import require_GET, require_POST

urlpatterns = [
    path("orders/<int:order_id>/email", require_POST(email_the_order)),
    path("runs/<str:execution_id>", require_GET(run_status)),
]

A log for each request

The journal of the SDK goes to one logger for the whole process. To find the records of one request, with_logger() gives each request a copy of the shared client that writes to a logger of its own. The copy sends through the same connections and records into the same span recorder; it owns no connections, and the shared client stays as it was — Logging.

  • RequestLog adds the id of the request your server is handling to every record, and keeps the fields each event of the SDK carries. A plain logging.LoggerAdapter before Python 3.13 replaces the extra of a call with its own, and the fields of the event would be lost. On 3.13 and later a plain adapter given merge_extra=True merges the two; the examples keep RequestLog, because the SDK runs on 3.11 and 3.12 too.

  • Your id goes under http_request_id. request_id of a record of the SDK is the id of the answer of Slen — the one its support finds the request by.

Do not wait in a handler

executions.wait() keeps the request of your caller open for as long as the run takes, and holds a thread or a task of the worker all that time. A handler hands the run to Slen with operations.execute_async() and answers 202 at once. The end of the run arrives as the webhook execution.finished — Webhooks — or a job of your queue waits for it. A read of the status, executions.get(), is one request, and may stand in a handler.

A run that a handler does wait for, operations.execute(), gets a budget of its own: timeout and retries of the call, not the minutes of the client — Retries and timeouts.

Close the client

async with AsyncSlen() as slen: in the lifespan closes the connections of the client when the worker stops, and close() does the same where there is no such block. A copy made by with_logger() owns no connections, so closing it closes nothing. An httpx.Client you passed as http_client stays open: it is yours to close.

A fork and --preload

A client made before a fork shares its connections with every child, and the processes then read and write the same sockets. gunicorn --preload, and any server that loads the application before it forks its workers, does that to a client built when a module is imported. Build the client in the worker, as both examples do. A client does not travel to another process either: pickling or copying it raises TypeError.

Where to go next