Adam Horwich

getting to the bottom of mongotop

While it’s easy to collect MongoDB metrics and ferry them away in your favourite local graphing service (or MongoDB’s MMS), there are some stats which are a little less trivial to collect, but still remain interesting. MongoDB provides a mongotop feature, that shows read and write latencies at the MongoDB Collection level. These stats aren’t collected by MMS, or even pymongo, but it’d be rather useful to know what collections are loaded the most in terms of read and write latency, and graph those to understand or even alert on usage.

top of the tops

So, I wrote a script that collects collection stats from monogtop and outputs them in a Graphite friendly fashion, ready to be integrated with Sensu.

So, a short and sweet summary of what we’re doing here. mongotop isn’t a fixed list of collection activity, it’s a list of the top 9(?!) active collections. Your MongoDB instance may contain several databases, with a large number of collections, consequently this list will not always contain the same collection. Nor will this be an exhaustive collection of metrics for all collections, only the top active ones. To address the issue of inconsistency in the top 9 collections, we simply take a count each time we see a collection recorded and take the average based on this number when we generate our metric. We can’t simply divide by the number of samples we’ve taken. In addition the script offers a flexible amount of samples, so you can catch more of an accurate snapshot of your collection activity.

latent communication

Whether you find these stats useful is entirely down to the nature of the problems you may foresee in your MongoDB usage. For example, it’s good to see which collections are busy when you’re trying to figure out where your bottlenecks are, or if you need to add in an index. But it won’t show you number of queries being made; so it’s best to use this in conjunction with mongostat and other metrics tools. It’s not amazingly useful if you have replication level issues though. The read/write latency seems to fall to 0 if you have heavy replication writes, which seems a little counterproductive (because the write lock prevents reads from being made), but hey, it’s MongoDB!

blog comments powered by Disqus