TS-50: Cloud economics

This technical standard focuses on financial decision-making for cloud deployments. It addresses the strategic questions of how cloud costs are structured, how to avoid unexpected expenditure, and how to choose between cloud-native and dedicated server architectures.

Managing costs is a critical aspect of cloud operations. Organizations need not only a clear understanding of their business-as-usual usage patterns and associated costs, but also the potential risks associated with "exploding costs" due to unexpected changes in throughput, storage, or other resource usage.

The guidance here is applicable to any organization running workloads on cloud platforms, regardless of scale or deployment model.

Much cloud spend is committed long before an invoice arrives, at the point somebody sets an availability, latency, or data-retention threshold. Those thresholds are quality requirements, and the standard requires each to be justified by weighing the value of meeting it against the cost of meeting it — see TS-1: Software requirements specification. A cost problem that traces back to an unjustified threshold is not solved by tuning the infrastructure.

For guidance on cloud platform engineering — including multi-account strategies and environment management — see TS-49: Cloud platform engineering.

Auto-scaling surge costs

The greater the degree of auto-scaling that is built-in to a cloud platform, the greater the risk of unexpected costs. Platform-as-a-service (PaaS) solutions, which tend to have uncapped auto-scaling by default, are particularly vulnerable to this risk.

Costs can explode from seemingly negligible charges. For example, a cloud provider may charge $0.023 per GB per month for hot (low latency) storage. This seems trivial. But consider the following scenario:

  • 10,000,000 users
  • 100 images storage allowance per user
  • 5MB maximum image size

10,000,000 users * 100 images * 5MB = 5PB = $115k/month

Strategies to avoid surge costs include:

  • Set limits on auto-scaling, eg. set concurrency limits for AWS Lambda functions, or instance limits for EC2 auto-scaling groups. Accept that some users may experience service unavailability during peak times, and communicate these limitations transparently to users, eg. via service level agreements (SLAs) and status pages.
  • Match revenue-per-user to costs-per-user before fully opening up auto-scaling. Do not enable auto-scaling until you have set up a paywall, subscription model, or other monetization strategy that scales with usage. Prioritize availability for paying customers over free-tier users.
  • Use inexpensive storage tiers and archive or delete data that no longer needs to be retained in a user-facing storage system.
  • For frequently-accessed data, use caching layers to reduce the number of requests to the underlying storage system, and implement compression on both data-at-rest and data-in-transit wherever possible.
  • Avoid excessive logging, especially in production. Use feature flags so that you can dynamically restrict logging to only the most important events. Also consider using techniques such as request sampling in tracing systems.
  • Prepare for denial-of-service (DoS and DDoS) attacks, eg. using Azure Frontdoor or AWS Shield, or Cloudflare.

The first of these strategies is a quality trade-off, not merely a configuration choice. Capping auto-scaling lowers the availability the system can offer under surge, in exchange for a bounded bill. Where an availability threshold has been specified, the cap MUST be set consistently with it, and a cap that cannot be reconciled with the specified threshold is a requirements problem to escalate rather than a value to tune. See TS-1: Software requirements specification for specifying and costing availability requirements, and TS-2: Software design qualities for resolving the underlying conflict between availability and cost.

Dedicated servers vs cloud-native architectures

A common mistake in cloud engineering is to reach for distributed, cloud-native architecture before it is actually needed.

Cloud-native architectures carry a significant baseline cost premium over running equivalent workloads on dedicated servers. Depending on the services used, the premium is typically in the range of 5–30x compared to renting a dedicated server from a hosting provider Serverless compute in particular (eg. AWS Lambda) is approximately 5–6x more expensive than an equivalent large cloud VM instance, and approximately 25x more expensive than an equivalent dedicated server from a budget hosting provider.

These premiums exist because cloud providers build in spare capacity to handle their customers' peak loads, and that cost is passed on.

The key principle is that the burstier the workload, the more justified cloud-native architecture becomes from a cost perspective. If a workload is highly unpredictable — with long periods of idleness punctuated by sudden large surges — serverless or auto-scaling architectures can be cost-effective, since you only pay for what you use. But for workloads with relatively steady, predictable traffic, paying the cloud premium provides little benefit.

As a rule of thumb, if a dedicated server would be kept above approximately 5% utilization almost all the time, it will likely be cheaper than equivalent serverless compute.

Modern servers have plenty of capacity to handle the workloads of many services. Modern servers are substantially more powerful than they were when distributed computing patterns became fashionable in the early 2010s. For most web services with under 10,000 queries per second (QPS), a single large server will be sufficient. Very few services exceed this threshold.[1]

Organizations SHOULD consider vertical scaling before adopting horizontal scaling or cloud-native distributed architectures. It is RECOMMENDED to scale vertically first (moving to a larger server) before scaling horizontally (adding more servers).

When horizontal scaling becomes necessary, using a small number of large servers will generally be more efficient than using a large fleet of small machines. That’s because there is a non-trivial coordination overhead associated with each node in a cluster.

The main drawback of single-server architecture compared to cloud-native architecture is availability. To mitigate this, it is RECOMMENDED to run a primary and a backup server in separate datacenters or cloud regions. For higher redundancy requirements, a 2x2 configuration — two servers in a primary datacenter and two in a secondary — is sufficient for most services.

To avoid correlated hardware failures (where a fault in one server increases the likelihood of a fault in another), primary and backup servers SHOULD use hardware from different manufacturing batches, and ideally from different models or vendors. This is especially important for storage devices.

Of course, there are trade-offs. Cloud service providers offer incredible levels of fault tolerance and availability, and they take care of many operational concerns such as security patching, DDoS protection, and data replication. Dedicated servers require much more hands-on management and maintenance – which can be expensive. Yet, for many web services, cost savings from using dedicated servers can be significant, and the availability can be sufficient with proper redundancy strategies in place.

The conventional wisdom is that cloud-native architectures are good because they let you scale up effortlessly. True. But the practical reality is that most web services have stable traffic patterns and predictable growth. This means scaling requirements can be foreseen and solutions planned well ahead of time.

Unless your traffic patterns are really bursty, the economic justifications for investing in auto-scaling will be, at best, weak. There may be other rational justifications for adopting cloud-native architecture, but cost savings is unlikely to be one of them.

Cost review

The strategies in Auto-scaling surge costs bound what a bill can reach, and Dedicated servers vs cloud-native architectures settles the deployment model at the point the decision is made. Neither tells an organization whether what it is paying today is still reasonable. Cloud spend drifts upward quietly — through retained data that nobody deletes, instances sized for a load profile that has since changed, and services adopted for one workload and left running for none — and none of that produces the kind of bill that prompts an investigation.

Organizations SHOULD therefore review cloud spend on a fixed schedule. Quarterly is sufficient for most; monthly is RECOMMENDED where spend is growing faster than the business it supports. A review that runs only in response to an alarming invoice cannot detect a cost that has been diverging gradually from the value it delivers, which is the more common failure.

Each review SHOULD establish three things:

  1. Where the money goes. Spend MUST be attributable to the service, feature, environment, or customer segment that incurred it. This is a design constraint on the platform, not a reporting exercise: an account and tagging scheme that does not distinguish these at the point of provisioning cannot be made to attribute spend retrospectively. See TS-49: Cloud platform engineering for the account and environment structure this depends on.
  2. How unit cost is moving, as distinct from total spend. Track cost per active user, per request, per gigabyte stored, or whichever unit the business actually meters. Total spend rising in step with usage is growth. Unit cost rising while usage is flat is a defect, and it is invisible in the total.
  3. How spend compares against a benchmark. Three benchmarks are useful, and they answer different questions: the previous period, which shows the trend; the budgeted figure, which shows whether the plan was wrong; and the cost of the same workload under an alternative deployment model, which shows whether the architecture is still justified.

A line item that exceeds its benchmark is a trigger to look for a solution. It is not, on its own, a defect.

Acting on a cost trigger

A benchmark breach obliges an investigation, not a change of technology. The cheapest responses are almost always configuration rather than architecture — storage tiering and retention, caching, compression, log volume, and scaling limits, all covered in Auto-scaling surge costs — and these SHOULD be exhausted before the cost is escalated into a technology-selection question. Re-platforming to solve a cost problem that a retention policy would have solved is an expensive way to arrive at the same bill.

Where those responses are exhausted and the cost remains disproportionate to the value delivered, the cost has become a legitimate trigger to re-evaluate the technology itself. That is a technology-selection decision, and it is made against the criteria in TS-6: Distributed system design, which covers the adoption triggers, the build-versus-buy evaluation, and the periodic re-evaluation of a stack. For the narrower case of a dependency or framework within a single application, see TS-5: Application architecture.

Where the response is build-versus-buy, the comparison MUST be made on total cost of ownership rather than on a vendor’s price against an infrastructure price. The build side of the comparison carries the engineering time to build the capability, the engineering time to operate it indefinitely, and the opportunity cost of the product work not done instead. A comparison that omits these will favor building almost every time, and will be wrong almost every time.

Validating cost estimates

A projected cost MUST be validated against a production-representative workload before the organization commits to the technology that incurs it. A pricing page states a unit rate. The bill is that rate multiplied by an access pattern, and it is the access pattern, not the rate, that estimates get wrong.

The gap between the two is routinely large enough to reverse a decision. PostHog’s evaluation of Amazon Elastic File System projected roughly US$300 per month and measured roughly US$600 per day in practice — sixty times the projected figure — which was sufficient reason to reject a technology that was otherwise a good technical fit.

The usual cause is a charging dimension the estimate did not price. Managed storage and database services commonly bill per operation, per request, or per byte scanned in addition to per gigabyte stored, and a workload’s operation count is far harder to estimate from first principles than its data volume. Where a service meters a dimension the estimate cannot confidently predict, that dimension MUST be measured under a representative load rather than assumed.

The corollary is that a cost problem can often be solved by moving a workload to a service whose charging dimensions suit its access pattern, without changing the architecture around it — moving infrequently read bulk data from a query engine to object storage, for example, where the data is written often and read rarely.

References

  • PostHog (2024). How We Choose Technologies. — The source for the quarterly cost-review cadence, cost-above-benchmark as a trigger, and the EFS estimate-versus-reality case in Cost review.
  1. For more details on what a single modern server is capable of, read "Use One Big Server" by Nima Badizadegan: https://specbranch.com/posts/one-big-server/ ↩