Marathon vs Aurora and their purposes

Question

Both Marathon and Aurora are built on Mesos and supposedly are engineered for running long running services. My questions are:

1. What are their differences? I have struggled in finding any good explanations regarding their key differences
2. Do these frameworks run anything that runs on Linux? For Marathon they state that it can run anything that "is executable in a shell" but this is sort of vague :)

Thanks!

Strychninism

And don't forget Singularity, from Hubspot : github.com/HubSpot/Singularity . And although limited to provisioning Docker containers, there is also Kubernetes github.com/mesosphere/kubernetes-mesos – Baptistery

Answer 1

Disclaimer: I am the VP of Apache Aurora, and have been the tech lead of the Aurora team at Twitter for ~5 years. My likely-biased opinions are my own and do not necessarily represent those of Twitter or the ASF.

Do these frameworks run anything that runs on Linux? For Marathon they state that it can run anything that "is executable in a shell" but this is sort of vague :)

Essentially, yes. Ultimately these systems are sophisticated machinery to execute shell code somewhere in a cluster :-)

What are their differences? I have struggled in finding any good explanations regarding their key differences

Aurora and Marathon do indeed offer similar feature sets, both being classified as "service schedulers". In other words, you hand us instructions for how to run your application servers, and we do our best to keep them up.

I'll offer some differences in broad strokes. When it comes to shortcomings mentioned in each, I think it's safe to say that the communities are aware and intend to fix them.

Ease of use

Aurora is not easy to install. It will likely feel like you are trailblazing while setting it up. It exposes a thrift API, which means you'll need a thrift client to interact with it programmatically (a REST-like API is coming, but is vaporware at the moment), or use our command line client. Aurora has a DSL for configuration which can be daunting, but allows you to easily share templates and common patterns as you use the system more.

Marathon, on the other hand, helps you to run 'Hello World' as quickly as possible. It has great docs to do this in many environments and there's little overhead to get going. It has a REST API, making it easier to adapt to custom tools. It uses JSON for configuration, which is easy to start with but more prone to cargo culting.

Targeted use cases

Aurora has always been designed to handle a large engineering organization. The clusters at Twitter have tens of thousands of machines and hundreds of engineers using them. It is critical to Twitter's business. As a result, we take our requirements of scale, stability, and security very seriously. We make sure to only condone features that we believe are trustworthy at scale in production (for example, we have our Docker support labeled as beta because of known issues with Docker itself and the Mesos-Docker integration). We also have features like preemption that make our clusters suitable for mixing business-critical services with prototypes and experiments.

I can't make any claim for or against Marathon's scalability. On the feature front, Marathon has build out features quickly, but this can feel bleeding edge in practice (Docker support is a good example). This is not always due to Marathon itself, but also layers down the stack. Marathon does not provide preemption.

Ownership

To some, ownership and governance of a project is important. It feel that in practice it does not define the openness of a project, but for some people/companies the legal fine print can be a deal-breaker.

• Marathon is owned by a company (Mesosphere)

To some, this is beneficial, to others is is not. It means that you can pay for support and features. It also means that there is something to be sold, and the project direction is ultimately decided by Mesosphere's interests.

• Aurora is owned by the Apache Software Foundation

This means it is subject to the governance model of the ASF, driven by the community. Aurora does not have paying customers, and there is not currently a software shop that you can pay for development.

tl;dr If you are just getting your feet wet with running services on Mesos, I would suggest Marathon as your first port of call. It will be easier for you to get running and poke around the ecosystem. If you are forming the 'private cloud strategy' for a company, I suggest seriously considering Aurora, as it is proven and specifically designed for that.

Amorette

Hi Bill, aurora looks like inspired by borg. Is there any roadmap documentation for aurora. – Waterfront

Answer 2

So I've been evaluating both and this is my summary.

Aurora

[+] also handles recurring jobs
[+] finer grained, extensive file-based configuration
[+] has namespaces so multiple environments can co-exist
[-] read-only UI, no official API
[~] file based configuration and cli based execution brings overhead (which can be justified with more extensive feature set)

Marathon

[+] very easy to setup and use
[+] UI that provides control and extensive API (even with features missing from UI at the moment)
[+] event bus to listen in on api calls
[-] handles only long-running jobs
[-] does not have separate deployment-run-cleanup steps, these if necessary need to be combined in a script of one-liner

Even though Aurora has better capabilities, I prefer Marathon due to Auroras complexity/overhead and lack of UI (for control) & API

Odom

Thanks Eren for the comparison! Have you tested HA and failover capabilities for both? To my understanding, if e.g. Nginx fails on one node, the exact runtime state is shifted to second node (if configured)? Or is the session/state lost during failover? – Strychninism

Marathon does not do this. If one 'app' fails, it's recreated from scratch according to its configuration. Aurora has step-by-step deployment so it may be able to carry over at least the sandbox and re-run it somewhere else. I have not checked this but with it's feature set, it should be able to (Marathon can not). – Revel

Answer 3

I have more experience with Marathon.

Ideological:

• Marathon is a relatively tested product that is used in production at AirBnB. Aurora is an early Apache project (so YMMV).
• Both are open source and active. Feel free to contribute pull requests or file issues!

Technical:

• Marathon doesn't schedule batch tasks or cron jobs
• Marathon has a friendly UI and better health indicators (in 0.8.x)

In regards to your second question, you can run any command or docker container, and Mesos will do the resource isolation for you. If you have 50% CentOS nodes and 50% Ubuntu nodes and you run a task that executes apt-get, the task will have a 50% chance of failure. Mesos and Marathon have no awareness of the actual machines.

Stome

FWIW Aurora is an early Apache project, but is not a new software project. It has been in development since early 2010. – Amorette

Using Mesos roles, you can tag your CentOS and Ubuntu nodes as such and the use Marathon constraints to schedule the job on the Role that makes sense. (github.com/mesosphere/marathon/blob/master/docs/docs/…) – Goldfilled

Constraints and attributes are the way to handle awareness of which OS. – Estrus

Answer 4

Disclaimer: I don't have hands-on experience with Aurora, only with Marathon.

ad Q1: In a nutshell Apache Aurora is capable of doing what Marathon + Chronos can provide, that is, schedule both long-running services and recurring (batch) jobs; see also Aurora user guide.

ad Q2: Yes, anything. Currently based on cgroups and Docker but hey, you can roll your own.

Daybreak

BTW, best video on Aurora usage I'm aware of is: 'Q&A with the Twitter Mesos Team', link: youtube.com/watch?v=1XYJGX_qZVU – Daybreak