Stateless design
A stateless design or system carries no information forward between interactions. Each request or invocation is handled independently, with everything it needs supplied fresh. It is the counterpart to stateful design.
In distributed systems, statelessness is a key to horizontal scaling. Any instance can handle any request, instances are interchangeable, and capacity is easy to add or remove. Because no instance holds state a caller depends on, failure of one instance just shifts work to another, which makes stateless designs fault-tolerant and friendly to load balancing and to active-active deployments where every replica serves live traffic.
In agent and LLM systems, the model is permanently stateless. Its parameters are frozen after training, and nothing you do at inference changes them – the model does not learn from corrections and does not remember being told the same thing yesterday. The feeling of continuity within a session is manufactured by the harness, which keeps the transcript and re-sends it with every request. The model is not remembering the conversation – it is re-reading it.
The practical consequence is that if you want something remembered across
sessions, you have to write it down somewhere the agent will read it back:
AGENTS.md files, memory systems, or
handoff artifacts. When the agent keeps making a
mistake you have corrected before, the question is not why it did not learn –
it cannot – but where that correction should be written down. This is the core
problem context engineering addresses.
As a general software-engineering trade-off, statelessness buys simplicity, scalability, and fault-tolerance at the cost of having to re-supply context every time (cost and latency) and the inability to build on prior interaction. FaaS functions are an extreme case: the platform assumes statelessness and may invoke a fresh instance for every call, so anything that must persist has to be written to an external store.