Over the few months I’ve been working here at MetaBroadcast I’ve worked with data from Atlas in various different projects, so have come to understand the API from a user perspective pretty well. More recently, however, I’ve had the chance to dive deeper, and experience modifying Atlas itself. This has given the team the opportunity to see what’s it’s like for someone unfamiliar with the inner workings to get know Atlas, and to find out what works and what doesn’t.
where do I start?
One of the first things I noticed is the number of repositories. There are currently 7 Java repos, ranging from Atlas’s Java client through the data model to the persistence layer, and more. The first stage of any addition to Atlas is figuring out where to start work. For instance, I wrote a new adapter to ingest data. The ingest code lives in the main atlas repository, but depending upon what data is ingested, there may be data model changes in atlas-model, which would cause knock-on effects in the persistence layer and also the simplification code (transforming from the stored ‘complex’ data model to the simplified form that’s output by the API). So basically, you’ve got to look carefully at what you’re doing, and watch out for changes that may creep across multiple repositories.
model all the things!
The data model in Atlas is not the simplest of things, even when looking at the simple model! Some of this complexity is just due to the sheer amount of different data types that we support, and some is due to Atlas’ gradual evolution over the years. As Fred has mentioned previously, we’re in the process of gradually reworking this, but these things take time, so when looking at atlas-model, I found that it can help to look at what inherits what, and draw a diagram of how the different classes interact—it’s not always obvious how the various data types are linked from a quick glance!
(don’t) run all the things!
Another point to bear in mind when trying to run Atlas, particularly Atlas processing, is that there are a lot of components. When writing my new site adapter, I found that running Atlas became a lot easier when all of the other remote site adapters were turned off, as I didn’t have to worry if I had all of the prerequisites for those, and could just focus on the code that I was writing. This is pretty simple to do—simply find the main properties file, and set all of the updaters.[remote site name].enabled properties to false.
if in doubt… ask!
As always with such problems, most of the issues that I encountered when working with Atlas were quickly solved by a quick discussion with either Fred or Tom. While I obviously have an advantage in that regard, those of you not working in the same office are not up the creek without a paddle—the Atlas mailing list is an excellent place to air any issues or questions you may have.