Architecture and workers
The workflow engine is a Go library inside your application, not a separate orchestration server. Applications share work by connecting to the same storage namespace. They do not register with a central application process or communicate through a worker-to-worker API.
Components
| Component | Responsibility |
|---|---|
client | Persist workflow starts, signals and cancellation; wait for results |
backend | Store histories and task queues, claim tasks and manage leases |
worker | Register definitions, poll queues, replay workflows and execute activities |
workflow | Deterministic scheduling, futures, signals, timers and concurrency |
diag | Optional read-only HTTP diagnostics over a backend |
A backend is a storage implementation, not whichever application happens to expose HTTP. A client-only process can enqueue work without running a worker. A worker-only process can execute that work without serving an API or UI.
Task lifecycle
- A client creates an instance with a registered workflow name, serializable arguments and a queue.
- A worker listening to that workflow queue claims a task through the backend.
- The worker replays recorded history and records new commands, such as scheduling an activity or waiting for a signal.
- An activity worker listening to the selected activity queue claims and executes the activity. Its result becomes a history event.
- The next workflow task replays that result and continues until the next wait or completion.
There is no fixed assignment of an entire workflow to one replica. Sticky execution and executor caching reduce replay overhead, but persisted history remains the recovery source. Another eligible worker can resume after ownership expires. Activity delivery is at least once; see replay and side effects.
Registration and identity
Register workflow and activity functions before calling Start. registry.WithName assigns an explicit definition name. Producers and eligible workers must agree on names, argument serialization and compatible workflow behavior. Registering a name is not authentication or tenant isolation.
Workflow instance IDs address durable work. Execution IDs distinguish runs, including continued executions. These are different from worker names.
backend.WithWorkerName sets the backend’s worker identity. Pass it through the selected backend’s WithBackendOptions. Give simultaneously running replicas distinct names. Defaults are generated by the backend; names identify lease owners, not service accounts or authorization roles.
Queues and worker roles
worker.New runs both workflow and activity workers. worker.NewWorkflowWorker and worker.NewActivityWorker split those roles between processes. Each process connects independently to the same backend namespace.
Set WorkflowQueues and ActivityQueues in the corresponding worker options. A new instance selects its queue through client.WorkflowInstanceOptions.Queue. workflow.ActivityOptions.Queue can route an activity elsewhere; by default it inherits the workflow’s queue. The queues sample shows separate routing. System queues are included by the worker implementation.
Every eligible worker must have the definitions it can receive. Queues select consumers; they do not enforce authorization. Isolate storage and credentials when applications must not see or execute each other’s work.
Concurrency and shutdown
WorkflowPollers and ActivityPollers control task polling. MaxParallelWorkflowTasks and MaxParallelActivityTasks control concurrent execution; zero means unlimited. Executor cache size and TTL affect workflow replay overhead, not durable history retention. An executor that expires or is evicted while a workflow task runs on it stays usable until that task finishes, and the next task for the instance replays its history in a new executor.
Keep heartbeat intervals comfortably below the corresponding backend lock timeout. A stalled process can lose its lease while its external operation continues, so cancellation and idempotency must also be handled by the activity.
Enable SingleWorkerMode only when exactly one worker instance consumes the backend. Leave it disabled for replicas and split-worker deployments.
Cancel the context passed to Start, call WaitForCompletion, then close storage owned by the application. A client result timeout only stops that client’s wait. It does not cancel the workflow.
Storage and HTTP boundaries
Use backend operations for connection ownership, migrations, persistence and retention. Use diagnostics to expose the UI. Any suitable application can host diagnostics for the same backend, but none is automatically elected to do so.
The library provides no application authentication layer, public workflow submission server or management cluster. The host application owns those boundaries.