Fred van den Driessche

the big atlas shake-up

Last year I wrote a brief post about where we intended take Atlas as a project including restructuring and reorganising the Git repositories in which it’s contained. In brief, Atlas is currently composed of seven repositories and we feel that with a little shuffling we can make development easier by making the most important parts of the code more pertinent and making hierarchies more consistent. We’re now about to start acting on some of those proposals so here’s a little more detail about what exactly we’re going to do.

out with the old…

The following repositories will be disappearing:

  • atlas-applications: There was an initial thought that applications in Atlas would be optional and while this may still be possible from a functional point of view it’s never really been true from a code point of view. The existing code will be split such that API end-points move into the main atlas repository while model and interfaces types move into atlas-model.

  • atlas-feeds: Since this repository contains both API end-points and background tasks it will be rolled up into the atlas repository.

  • atlas-search: We’re in the process of adopting elasticsearch for our search index so the search server will soon be obsolete.

  • atlas-persistence: All of the persistence-related interfaces will be moved into atlas-model. We’ll also be moving out all of the work we’ve been doing with Cassandra and elasticsearch. This should leave atlas-persistence as a MongoDB-specfic of Atlas’ persistence module. We may spare it for while but we probably won’t be doing much development on it, especially once we move fully to Cassandra.

…in with the new

The existing Cassandra and elasticsearch work will find homes in new repositories, atlas-cassandra and atlas-elasticsearch respectively. The idea is these are technology-specific implementations of Atlas’ persistence interfaces, they should contain no (public) interfaces themselves.

atlas-model will house all the model types, as it does currently, along with all of the core interface types: stores, indices etc. The intent is it becomes the main dependency of all other project components and as such it will also contain common utilities.

Each of the remote site adapters, the processes used to bring data into Atlas, will be spun out of atlas to live on their own, permitting each adapter to evolve independently of every other, depending solely on atlas-model. Build times for atlas should also improve since the adapter tests will also move so we don’t have to run the tests for adapters A to Y just because adapter Z was updated. The greater flexibility provided should also encourage more in-depth integration tests for each of the adapters.

For now the Atlas API and Processing servers will stay together in atlas as they have enough in common that it’s not worth separating them. That said, as Atlas continues to evolve and grow into its new architecture we may eventually see fit to pull the two apart.

We hope this reorganisation will lead to a more easily understood codebase. If you feel there’s a better way everything could be organised, or if you have any other questions or comments, then please let us know, either in the comments below, or on the mailing list.

blog comments powered by Disqus