Garry Wilson

jumping on a moving train

For the second time during my stint at MetaBroadcast, we’re embarking on a project to migrate all of our infrastructure to a new setup.

First time around, we moved to Amazon’s VPC, which wasn’t around at the time MetaBroadcast’s AWS account was started. In doing so we changed from using scripts to manually start instances, to having launch configurations and autoscaling groups for every instance. With that, we gained the resilience of having instances that would be recreated automatically, and the simplicity of replacing all that scripting with a few clicks in the AWS console.

Although the migration took its time, and came with its own set of unforeseen problems, we had the benefit of moving from what we knew (EC2 Classic) to something that had then been released for a while, was well documented, and – as it was our own private network – mapped closely to what you’d expect a network to do.

all aboard the open-source express

This time, in moving to Kubernetes, we’re moving towards what is still a young project. Although it has proven itself to be a steady and reliable one, it suffers from many of the problems that most new tech does; documentation isn’t always there (or reliable), best practices aren’t obvious, and features change during development.

We spent quite a bit of time last year examining the differences between Kubernetes and Marathon, running trials of each and eventually settling on Kubernetes. It represents the best set of features, and obviously it doesn’t hurt that it’s backed by Google.

moving the goalposts

There have been a number of features we’ve been waiting for in Kubernetes, the main being the introduction of Pet Sets. We’ll need it in order to move our data stores (Mongo, Cassandra, Kafka, ElasticSearch) to the magical containerised world of K8. Although it’s currently there in beta form, we’re looking forward to playing with it more when it’s finally released in version 1.4.

Problems arise when we start to depend on roadmap features which may get bumped or delayed. The release of 1.3 itself was delayed, which we’d been waiting for in order to make the cluster highly avaialble.

We’d been hoping to switch with version 1.3, but a bug limiting the length of Kubernetes’s service DNS means we’re holding off to launch once 1.4 is released. That should happen next week. Then, post-switch, we can start playing with migrating those data stores from being autoscaling instances, to being Kubernetes Pet Sets.

what to consider

The main thing to take into account is what features we need in order to switch. We want to be able to build an entire copy of our infra in Kubernetes, have both running side by side, and change only when we’re happy it’s all stable and working.

One of the most fun parts of my time here at MetaBroadcast is that we’re always happy to play with new and emerging projects, both good and bad, to try and find the best fit for our needs. It’s never going to be seamless, but then, that’s half the fun of the job.

If you enjoyed the read, drop us a comment below or share the article, follow us on Twitter or subscribe to our #MetaBeers newsletter. Before you go, grab a PDF of the article, and let us know if it's time we worked together.

blog comments powered by Disqus