User Tools

Site Tools


sig:sysadmin:projects:project18

This is an old revision of the document!




Project Member(s)

  • Jeff Gensler (jgensl2@uic.edu, jgensler8@gmail.com)

Purpose

Provide an easy (enough) to use platform for application developers and data scientists to deploy applications (Node.js/Python/Ruby) or “Big Data” (MapReduce/Spark/Flint) jobs without having to set anything up locally.

Use cases:

ACM member A creates a Node.js App that needs MongoDB. If ACM member A's app becomes a hit on campus, A needs to scale the database, load balancing, etc to accommodate the new traffic.

ACM member B takes a Machine Learning class and decides they will use Apache Spark for analysis. B realizes that one standalone container for testing Spark is not powerful enough for the job and wishes to deploy a set of standalone containers (as opposed to YARN/Mesos on top of CoreOS) to tackle their job.

Resources

Progress

We have chosen Docker as the method of containment for such applications and CoreOS as a means of holding and scaling these containers. Typically, a user would write a systemd unit file that would typically download a docker container, name the container, and deploy it across the cluster, lettings CoreOS balance the load. While this granularity is fine for a select few, it will be too much work for someone that wants to quickly deploy their application.

Orchestration/Deployment tools for CoreOS + Docker

Deis

Deis appears to be the easiest way for an single application to be deployed. Deis only focuses on the application container and not on the database container. It is unclear how to deploy a database container so that deis will deploy it on the same server as an application container. Deis appears to have the most support out of all of the products looked at. This product would be great for a simple application developer but would fail for our fellow data scientists.

Fig2CoreOS

Fig2CoreOS produces the exact end result of the project in that it allows a highly customize-able, highly-scale-able, and isolated deployment of an application. However, this still doesn't solve the ease of use for a non-sysadmin to deploy an application. I think fig is intuitive enough, but I am not sure about support from the outside world not.

Rancher

Rancher uses Github as an ACL but we can “simply” change the underlying technology to point to a (our) Gitlab server. This tool is really neat because it allows a group of people to modify a set of containers. This seems like a promising tool for monitoring and quick modification of containers, and interaction with the REST API would provide an easy way for scripts to manually scale applications. This seems like a solid fit for easy to deploy applications.

Shipyard

This product uses certs and publicly facing docker daemons which sounds pretty dangerous (check out docker's documentation for that one). The product doesn't really look all that different than Rancher and has a bit more logic (like RethinkDB over etcd).

Orchestration/Deployment tools for Mesos + Docker

Mesosphere

For running Docker containers on Mesos.

Marathon

This might be a suitable solution for the data scientist…

sig/sysadmin/projects/project18.1426986592.txt.gz · Last modified: 2021/05/02 21:36 (external edit)