Hello! We’ve recently been dealing with bootstrapping our largest dataset from Atlas Owl into the new Atlas Deer. Unsurprisingly this led to a need for a larger amount of resource than we would typically run. Furthermore we would need to run this extra resource over the period of a couple of days. All of our infrastructure and services run on the Amazon Web Services cloud. This means we are able to scale our services and infrastructure relatively painlessly through good use of launch configurations and auto-scaling groups. So just up our instance size and be done with it right?
well yes, but
We can do one better. We can use spot instances to provide this extra resource at a lower rate than the on-demand instances. If you don’t know, a spot instance is identical to a normal EC2 instance, but at a lower price and with the catch that if the bid you place on the instance is out matched, the instance will be terminated automatically.
In this case we went for a r3.4xlarge, which is normally priced at $1.4, however with a bid price of $0.5 we would pay a lot less. This doesn’t seem like a great saving, but we also scaled our ElasticSearch cluster up at the same time to handle the same increase in load. Again we used spot instances for this, allowing us to go from 3 m3.medium to 12 m3.larges for only a small increase in cost.
how does one do this
Above is a graph showing the bid price over time for an r3.4xlarge in the 3 EU-West availability zones. The bid price is the current price one would have to bid no less than in order to acquire an instance of this type. Here at MetaBroadcast we try to use a bid price that would result in a spot instance being terminated due to not meeting its bid price at most once a month. We find this gives us a good balance between making an actual saving in cost, while still retaining a high level of stability for our instances.
We use spot instances pretty extensively here, for example our Cassandra cluster has a number of core non-spot instances running inside of it, backed up by several more spot instances that are free to come and go as the market price for the instance drops above and below our maximum spend on them. They are a great source of nice–to–have extra resource that isn’t critical to healthy operation, at a lower price than normal.
If you enjoyed the read, drop us a comment below or share the article, follow us on Twitter or subscribe to our #MetaBeers newsletter. Before you go, grab a PDF of the article, and let us know if it’s time we worked together.