This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
sig:sysadmin:projects:project18 [2015/03/22 03:03] jgensler8 |
sig:sysadmin:projects:project18 [2021/05/02 21:36] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Containerized Deployments and Applications (Cluster Project)====== | ||
====== Project Member(s) ====== | ====== Project Member(s) ====== | ||
- | * Jeff Gensler (jgensl2@uic.edu, jgensler8@gmail.com) | + | * Chase Lee |
====== Purpose ====== | ====== Purpose ====== | ||
Line 11: | Line 12: | ||
ACM member A creates a Node.js App that needs MongoDB. If ACM member A's app becomes a hit on campus, A needs to scale the database, load balancing, etc to accommodate the new traffic. | ACM member A creates a Node.js App that needs MongoDB. If ACM member A's app becomes a hit on campus, A needs to scale the database, load balancing, etc to accommodate the new traffic. | ||
- | ACM member B takes a Machine Learning class and decides they will use Apache Spark for analysis. B realizes that one standalone container for testing Spark is not powerful enough for the job and wishes to deploy a set of standalone containers (as opposed to YARN/Mesos on top of CoreOS) to tackle their job. | + | ACM member B takes a Machine Learning class and decides they will use Apache Spark for analysis. B realizes that one standalone container for testing Spark is not powerful enough for the job and wishes to deploy a set of standalone containers (as in, YARN or Mesos on top of CoreOS/Docker) to tackle their job. |
====== Resources====== | ====== Resources====== | ||
Line 36: | Line 37: | ||
[[http://mesosphere.github.io/marathon/]] | [[http://mesosphere.github.io/marathon/]] | ||
+ | |||
+ | [[https://tsuru.io/]] | ||
+ | |||
+ | [[https://github.com/jwilder/docker-gen]] | ||
====== Progress ====== | ====== Progress ====== | ||
- | We have chosen Docker as the method of containment for such applications and CoreOS as a means of holding and scaling these containers. Typically, a user would write a systemd unit file that would typically download a docker container, name the container, and deploy it across the cluster, lettings CoreOS balance the load. While this granularity is fine for a select few, it will be too much work for someone that wants to quickly deploy their application. | + | We will be experimenting with both docker and rkt container engines. The current cluster will most likely be reinstalled. We have briefly discussed the project over the weeks via listserv and SIG meetings. I think we will be settling on a master node to orchestrate the cluster. The master node will also host a master etcd server and a kubernetes-head server. The remaining nodes will be worker nodes. |
- | ====== Orchestration/Deployment tools for CoreOS + Docker ====== | + | ===== Roadmap ===== |
+ | |||
+ | - Create cloud-config base for master and worker nodes. | ||
+ | * Note to account for: | ||
+ | * Bonded interface | ||
+ | * Internal DHCP server bonded interface | ||
+ | * external traffic interface | ||
+ | * Starting master etcd instance on master node for other machines to sync off of. | ||
+ | * Starting fleetctl | ||
+ | - Install Kubernetes | ||
+ | - Install Test load balancers | ||
+ | - Install whatever else... | ||
+ | |||
+ | ===== Orchestration/Deployment tools for CoreOS ===== | ||
+ | |||
+ | ** Kubernetes ** | ||
+ | |||
+ | Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure. Kubernetes is developed by Google. They have the concept of pods, which are a grouping of containers to run a given service. It seems to have a lot of support, as they are using this system on Google's Cloud Platform. | ||
+ | |||
+ | ** rkt (rock-it) ** | ||
+ | |||
+ | rkt is the next-generation container manager for Linux clusters. Actively developed from the people who develop CoreOS, rkt is designed for security, simplicity, and composability within modern cluster architectures, rkt discovers, verifies, fetches, and executes application containers with pluggable isolation. rkt can run the same container with varying degrees of protection, from lightweight, OS-level namespace and capabilities isolation to heavier, VM-level hardware virtualization. | ||
+ | |||
+ | ===== Orchestration/Deployment tools for CoreOS + Docker (Old) ===== | ||
**Deis** | **Deis** | ||
Line 53: | Line 81: | ||
**Rancher** | **Rancher** | ||
- | Rancher uses Github as an ACL but we can "simply" change the underlying technology to point to a (our) Gitlab server. This tool is really neat because it allows a //group// of people to modify a set of containers. This seems like a promising tool for monitoring and quick modification of containers, and interaction with the REST API would provide an easy way for scripts to manually scale applications. This seems like a solid fit for easy to deploy applications. | + | Rancher uses Github as an ACL but we can "simply" change the underlying technology to point to a (our) Gitlab server. This tool is really neat because it allows a //group// of people to modify a set of containers. This seems like a promising tool for monitoring and quick modification of containers, and interaction with the REST API would provide an easy way for scripts to manually scale applications. This seems like a solid fit for application with minimal one-time configurations and not so helpful for elastic services. |
**Shipyard** | **Shipyard** | ||
Line 60: | Line 88: | ||
- | ====== Orchestration/Deployment tools for Mesos + Docker ====== | + | ===== Orchestration/Deployment tools for Mesos + Docker ===== |
**Mesosphere** | **Mesosphere** | ||
- | For running Docker containers on Mesos. | + | We can actually deploy this on top of CoreOS. If we run a global service, fleetctl can actively maintain the number of nodes needed for a given Mesos VM. |
**Marathon** | **Marathon** | ||
- | This might be a suitable solution for the data scientist... | + | This is a WebUI interface for administrating the Apache Mesos Cluster. |
- | ====== Easy deployment tools for Docker ====== | + | ===== Easy deployment tools for Docker (old) ===== |
**Dokku OR Dokku-alt** | **Dokku OR Dokku-alt** | ||
This will give up a Heroku like deployment for a simple application. Dokku-alt will give us the ease of deployment of MariaDB, PostgreSQL, MongoDB or Redis databases. However, the lack of a multi-tenant nature of Dokku is the problem that Deis solves. See ([[http://serverfault.com/questions/640038/scaling-out-dokku-infrastructure | this SO question]]) | This will give up a Heroku like deployment for a simple application. Dokku-alt will give us the ease of deployment of MariaDB, PostgreSQL, MongoDB or Redis databases. However, the lack of a multi-tenant nature of Dokku is the problem that Deis solves. See ([[http://serverfault.com/questions/640038/scaling-out-dokku-infrastructure | this SO question]]) | ||
+ | |||
+ | **Tsuru** | ||
+ | |||
+ | Open Source PaaS solution. Might not have so much support later on but integrates Heroku buildpacks (like Dokku, etc) so ease of use for developers. | ||
+ | |||
+ | |||
+ | ===== Multi-Tenant ===== | ||
+ | |||
+ | [[http://techs.enovance.com/7104/multi-tenant-docker-with-openstack-heat | OpenStack w/ Heat]] |