Requests

Everything a request can be told to do. Every method returns *Request, so it all chains.

Contents

Client and request lifecycle

A Client is reusable and safe to hold for the process lifetime.

A Request is not consumed by running it. Building is single-goroutine (the With* setters write without synchronization), but execution never writes back, so a fully-built Request can be run repeatedly, and concurrently from several goroutines. Each call snapshots the assembled transcript into private run state.

client := elelem.New(
	driver,
	elelem.WithDefaultModel(elelem.Model{ID: "some-model", ContextSize: 200_000}),
	elelem.WithClientTokenCounter(myCounter), // optional
)

request := elelem.NewRequest(client)

WithDefaultModel is worth setting. Without it a request that names no model falls back to a bare Model{}, which has no ID, so Run refuses it with ErrInvalidRequest. A model with an ID but no ContextSize silently disables every context-size check. Nothing errors; the budget just stops existing.

client.Driver() hands back the underlying driver if you want to compose your own decorators around it. It returns nil for a nil Client rather than panicking, so a nil check at every call site isn’t needed.

Terminal calls

Call Does
Run(ctx) Sends the request. Tools go out if any are configured.
RunInto(ctx, &dst) Structured output. See structured-output.md.

Two launchers, and the only difference is whether you want a typed value back. Whether tools are sent is not a choice made at the call site. A request that carries tools sends them, one that doesn’t, doesn’t:

request.Run(ctx)                                      // no tools configured, none sent
request.WithTools(tools).Run(ctx)                     // manual: you execute
request.WithTools(tools).WithAutoToolCalls().Run(ctx) // automatic loop

To send no tools, configure none. RunInto is the one exception, and it is not a toggle either: a structured turn always suppresses tools, because a typed object and a tool list are competing answers to the same turn.

Watching the reply arrive piece by piece is a callback, not a launcher (OnDelta / OnText / OnReasoning, see callbacks.md). Whether the HTTP call itself streams is WithStreaming, below.

Manual driving is the default. A Run without WithAutoToolCalls returns as soon as the model asks for a tool, handing you Response.ToolCalls and Response.ExecuteToolCalls. Call the latter exactly once to advance a round. That is the right mode when a human has to approve a call, or when you want to decide per call; see tools.md.

WithAutoToolCalls() is what makes the engine run the loop itself until the model stops asking for tools or MaxRounds is hit.

Streaming

elelem streams by default: the driver opens a streaming request and the engine assembles the reply from the deltas as they land.

Some OpenAI-compatible and Anthropic-compatible backends cannot serve that. An async job queue in front of the model has nowhere to put a token stream, and says so. WithStreaming(false) sends the same request with streaming off:

client := elelem.New(driver, elelem.WithStreaming(false)) // every request
request.WithStreaming(false)                              // just this one

The request-level setting wins over the client-level one, in both directions.

Turning streaming off changes the transport, not the API. The driver feeds the finished response through the same delta callback the streaming path uses, so OnDelta / OnText / OnReasoning still fire, tool calls still assemble, and the Response is the same. It just arrives in one piece instead of many.

A driver that cannot stream at all reports Capabilities.StreamingUnsupported, and then the non-streaming path is taken no matter what was asked for. Note the negative phrasing: the zero value means streaming works, so a driver that never sets the field keeps the default rather than silently moving its traffic onto a different transport.

One provider-side limit is worth knowing before you disable streaming: the Anthropic SDK refuses a non-streaming request whose max_tokens implies a run longer than ten minutes, and it refuses locally without sending anything. The Anthropic driver surfaces that as ErrStreamingRequired so it can be told apart from a transport failure.

The prompt

The whole conversation (system message, history, and this turn’s question) is one Prompt, handed over in one call:

request.WithPrompt(elelem.NewPrompt().
	WithSystem("You are a concise operations assistant.").
	AppendSystem("Never speculate about root cause.").
	WithHistory(previousResponse.Messages).
	UserText("Summarize the current incident state."))

Prompt is immutable, so one can be built once and run repeatedly, against several models, from several goroutines. WithPrompt REPLACES whatever the request held. Build the conversation on the prompt, not by calling it twice.

Images, audio and documents ride on a user message as content parts:

elelem.NewPrompt().User(
	elelem.TextOf("What is going on in this screenshot?"),
	elelem.ImageBytes(png, elelem.MediaTypePNG),
)

Every builder method, the origin rules, the content-part constructors, and what each provider actually accepts are in prompts.md.

Model 

request.WithModel(elelem.Model{
	ID:          "some-model",
	ContextSize: 200_000,
	Pricing:     elelem.ModelPricing{ /* ... */ },
})

ContextSize and Pricing are yours to supply. They are NOT capabilities and are not discovered. ContextSize drives budgeting; Pricing drives Response.Cost, which is 0 when pricing is absent. 0 means unknown, not free.

MaxOutputTokens is the largest max_tokens the provider accepts for the model, reasoning included, and 0 means unknown. The Z.ai Coding catalog fills it for GLM 5.3 and GLM 5.3 Flash. elelem never sends it on its own. Pass it to WithMaxOutputTokens to cap a reply at the model’s ceiling.

Ask the driver what a model can do:

capabilities := client.Driver().Capabilities(model)
models, err := client.Driver().ListModels(ctx)

Generation parameters

Set them individually, or hand over a whole GenerationParams block with WithGenerationParams (which clones it, so your copy stays yours).

Method Type
WithTemperature(v) float64
WithTopP(v) float64
WithSeed(v) int64
WithStop(vs...) ...string (the slice is copied)
WithFrequencyPenalty(v) float64
WithPresencePenalty(v) float64
WithMaxOutputTokens(v) int64
WithReasoningEffort(v) ReasoningEffort

Unsupported settings fail locally, before any network call, rather than being silently dropped or bounced back by the provider a round later.

Reasoning effort is gated per model on both providers, and the levels a model accepts are not always contiguous. A model may take max while rejecting xhigh. Build against the model’s own levels rather than the constant list:

request.WithReasoningEffort(model.ReasoningLevelHigh())

ReasoningLevelMin/Low/Medium/High/Max() each fall back to a sane default when the model declares nothing.

Provider-specific parameters

The escape hatch for anything portable types don’t cover:

request.
	WithParam("logit_bias", map[string]int{"50256": -100}).
	WithParams(map[string]any{"user": "tenant-42"})

The OpenAI driver sends these as top-level body fields as-is, and drops a key naming a field it builds itself (messages, model, stream, tools, tool_choice) rather than letting it overwrite what the engine set. The Anthropic driver accepts only top_k and refuses any other key with its ErrUnsupportedParameter.

The tool loop

See tools.md for the full picture. The request-side methods:

request.
	WithTools(toolSet).                 // or WithTool(t), or WithToolProvider(fn)
	WithAutoToolCalls().                // opt into the engine driving the loop
	WithToolChoiceMode(elelem.ToolChoiceModeAuto).
	WithParallelToolCalls(true).
	WithForceFinalAnswer(true)          // withhold tools on the last allowed round

WithToolChoiceMode takes ToolChoiceModeAuto, ToolChoiceModeNone or ToolChoiceModeRequired. Forcing one tool needs its name, so it goes through WithToolChoice(elelem.ToolChoiceTool("name")); a name that is not in the set, or a driver without tool choice support, is refused locally with ErrInvalidRequest.

WithToolProvider(func(ctx) (*ToolSet, error)) resolves a fresh set before every round, for catalogs or authorization that can change mid-conversation.

WithForceFinalAnswer is the difference between “hit max rounds and returned a tool call nobody will execute” and “hit max rounds and answered.”

Queued user messages

UserMessageQueue accepts user input while an automatic agent loop is active. The queue is bounded, mutex-protected, and accepts only user-role messages. It rejects assistant, tool, system, and injection fields, so incoming input cannot forge transcript protocol state.

queue, err := elelem.NewUserMessageQueue(16)
if err != nil {
	return err
}

request := elelem.NewRequest(client).
	WithPrompt(prompt).
	WithTools(tools).
	WithUserMessageQueue(queue).
	WithAutoToolCalls()

// This is safe from a request or websocket goroutine while Run is active.
if err := queue.EnqueueText("Stop and explain the last action."); err != nil {
	return err
}

The engine drains queued input before its first provider request (automatic loop only), after every set of tool results and injections, and after a terminal assistant answer. It never changes a provider request that is already in flight. Pending input is FIFO and becomes normal MessageOriginTurn transcript entries.

WithAutoToolCalls is required to continue after a terminal answer. A manual loop still carries queued input into the provider request after you call ExecuteToolCalls, but it never starts a new terminal round by itself. A run at WithMaxRounds leaves pending input queued for a later Run.

Bounds

Some of these ship with a default; the rest are genuinely unbounded until you set them. The difference matters. Assuming a bound exists when it does not is how a stream runs forever, and assuming none exists when one does is how a tool loop stops at twelve rounds for no visible reason.

Method Bounds Default
WithTimeout(d) The whole request. The only thing stopping an endless stream. none (unbounded)
WithToolTimeout(d) Each tool’s entire run, hooks included, not just the handler. none (unbounded)
WithMaxToolResultTokens(n) Each tool result, measured on the finished string including the truncation marker. none (unbounded)
WithMaxContextTokens(n) The whole context, prompt and reply. The prompt gets this minus the output reserve. See history.md. the model’s ContextSize
WithMaxRounds(n) Tool-loop rounds. Exceeding it returns ErrMaxRoundsExceeded. 12
WithMaxConcurrentTools(n) Goroutines as well as running handlers. 8
WithOutputReserveTokens(n) Room kept for the reply, taken off the context limit. An explicit 0 keeps none. MaxOutputTokens if set, else 4096
WithForceFinalAnswer(v) Withholds tools on the last allowed round. true
WithTokenCounter(c) Overrides the counter for this request only. request → client → driver → package default

The prompt budget is the context limit, WithMaxContextTokens or else Model.ContextSize, minus the output reserve. Run refuses an explicit limit the reserve fills with ErrInvalidRequest. With no explicit limit, limiting stops in two cases: when the model carries no ContextSize, and when the reserve is greater than or equal to it. With the default 4096 reserve, a Model{ContextSize: 4096} resolves to no budget at all, silently.

Note WithForceFinalAnswer is already on. The examples above pass true explicitly for clarity; pass false if you would rather hit MaxRounds with an unexecuted tool call than have the model forced to answer.

Check before you send:

reached, err := request.IsTokenLimitReached()

It returns false when no budget is resolvable. A model with no ContextSize and no explicit cap can’t report a limit it has no way to compute.

The Response 

type Response struct {
	Text       string
	Reasoning  string
	ToolCalls  []ToolCall
	Usage      Usage        // whole run, already summed across rounds
	Messages   []Message    // full transcript, deep-copied
	Injections []MessageInjection
	Cost       float64      // 0 = unknown, not free
	Model      string
	FinishReason FinishReason
	ExecuteToolCalls func(context.Context, ...ToolCallDecision) (*Response, error)
}

Usage is the run total, so don’t sum it yourself across rounds; you’ll double-count. Usage.BilledTotalTokens() adds the tokens burned by failed retry attempts, and Usage.Total counts only the attempt that succeeded. Use the former for cost, the latter for context. Usage.ResponseID is the provider’s identifier for the last call, when the provider reports one. Quote it in a support request to the provider.