we write about the things we build and the things we consume
Adam Millican

voila a-z: o is for optimization

This blog post is about how we optimize calls and retrieval speeds of data using annotations.

Let’s say we didn’t use annotations to optimise voila, what would things look like then? Well, whenever anyone made a call for a specific piece of data, they would get everything!

First of all there would be far more database calls happening, which can slow things, and is expensive.

Second, whatever service is retrieving these calls now has far more to process through, and if that service happens to be me debugging something, it’ll take longer to find the piece of information that’s relevant to me.

so how do annotations work?

When a call reaches Voila, some code interprets the parameters of the call and figures out what needs to be accessed from the database, how it should be processed, and then how it should be presented and returned. The key thing that annotations do is provide a trade-off between database access and actual processing. With a small amount of extra processing logic to determine which exact pieces of data need to be accessed from the database, as defined by the annotations in the call, the code can reduce the database access necessary, and also decrease the size of unused data being sent.

To put it in simple terms, not using annotations is like going to a restaurant and ordering everything off the menu, so that it is brought to your table, and then only eating one slice of cake, discarding the rest. Sure, some days you might be in the mood for one of everything on the menu, and that might be an effective way of getting it all sent to you as fast as possible, but generally if you’re just going to eat the cake then just order the cake.

In this bizarre analogy the cooking, waiting time and effort is the database, and you eating cake is the service extracting the useful piece of data out of the call The waste of food that you don’t eat is actually just a small amount of data being sent across the internet unnecessarily, but it still kind of works.

example annotations usage

Let’s say I’m searching Voila for Game of Thrones. A nice quick search with the annotation ‘description’ returns several entries of the following size (7 to be exact):

  • id: “wybj”
  • type: “item”
  • title: “Game of Thrones Preview”
  • description: “A look ahead to the fantasy series, starring Sean Bean and Lena Headey.”
  • specialization: “tv”

Which after copying the JSON into Sublime Text is counted at 53 lines.

Now, let’s add the annotation ‘broadcasts’. Immediately I’m noticing that my browser JSON viewing extension is taking a few seconds to process everything. There’s now an extra field called ‘broadcasts’ that contains so much nested data that Sublime Text tells me I received a 3272 line long JSON.

That works out as over 60 times larger, taking far more time to access from the database, send, receive, and process.

Now, a way you could use annotations would be to make two calls. One to retrieve the list of entries, from which you select the one you actually want, and then another one to grab the broadcasts from this particular item, already cutting the size of the data seven fold.

But that’s not all, there’s far more data available than just broadcasts and description. Imagine if every call grabbed every piece of related data for every item found by the search, it would be even larger.

So that’s why we use annotations for optimisation. What do you think? Have you seen something similar before? Let us know in the comments.



If you enjoyed the read, drop us a comment below or share the article, follow us on Twitter or subscribe to our #MetaBeers newsletter. Before you go, grab a PDF of the article, and let us know if it’s time we worked together.

blog comments powered by Disqus
sign up to #metabeers
slideshow