bk99.de entertain the web since 1997

Amazon S3: storage for the internet

Summary

On 13 March 2006 Amazon announced the Simple Storage Service, S3 for short: storage that developers can use in any amount from anywhere through a simple web interface. Amazon promised the same scalable and inexpensive infrastructure on which it ran its own websites. S3 became a cornerstone of cloud computing, and its interface became the standard that many other storage systems imitate today.

What S3 did differently

Until then, storage was bought or rented as disks, servers or web space with fixed limits. S3 instead sold objects in buckets, addressed over HTTP and billed by volume and transfer. Anyone wanting to store a file needed no server, just a key and an address.

The simple interface became a success far beyond Amazon: many storage products, including free ones such as MinIO or Ceph with its object gateway, speak the S3 API today. How dependent the net had become was shown on 28 February 2017: a typo during maintenance took S3 down in the US-East-1 region for hours, and with it numerous websites and services.

Ideas

  • Storage as a service replaces storage as a device.
  • A simple interface can become an industry standard.
  • Pay-per-use billing lowers the barrier to entry.
  • Centralisation makes outages bigger.

Insights

  • S3 showed that infrastructure can be sold like software; EC2 followed the same year, and together they became the business that carries a large part of today’s internet.
  • The 2017 outage was the flip side: whoever puts everything in one region of one cloud inherits its failures.

Facts

  • Announced on 13 March 2006.
  • According to the announcement, S3 uses the same infrastructure Amazon used to run its own websites.
  • On 28 February 2017 S3 in US-East-1 failed for several hours after an incorrect maintenance command.

References

Remarks

  • The announcement is short; prices and technical details were in the product description at the time.

Recommendations

  • Do not keep important data in only one region and with only one provider.
  • For your own object storage, choose S3-compatible software so that tools stay interchangeable.

Read the announcement

Search the Web Archive