Webserver
The user facing portions of Platform.X are running on a single virtual machine, hosted on Amazon EC2, which we simply call the Webserver. This is where web applications are actually served from.
In the webops world, scaling vertically simply means add more power to a single machine as the load on the services hosted by that machine increase. Scaling horizontally generally means adding more machines to a cluster of similar machines as the load on the services hosted by the cluster increase. It is also generally true that scaling horizontally is more complex than scaling vertically.
In our case, we want to scale the Webserver vertically for as long as we practically can. That is, we'll keep reserving more and more powerful EC2 machine images to host the Webserver until the cost of the big machine is significantly more than the cost of running a cluster of smaller machines, or the load is so great that there is no single machine big enough to handle it.
Although we want to design for the eventuality of horizontal scalability, by scaling vertically for as long as possible we avoid a lot of complex issues in the early days of Platform.X.
Webserver component and service diagram
At the core of Platform.X is Enginemill , a web development framework and SDK built on Node.js. Since applications running in Platform.X are required to be sandboxed, they are wrapped together with Enginemill by a environment wrapper we call Apprunner. The sandboxing provided by apprunner is important for creating a platform capable of cotenancy.
Currently Apprunner is just a Node.js executable script that uses the
Enginemill libraries to run an application. An application process originates
in the apprunner script, which can be found in the
web_server
bin/ directory. The apprunner.js script then calls on the Enginemill library
to execute and load the application. So each web application can be thought of
as a bundle consisting of Apprunner + Enginemill + application code.
Eventually AppRunner will evolve into a full fledged component as our sandboxing requirements get more complex. In our current form, we are not doing any real sandboxing, so AppRunner is just a placeholder for future development.
See Webapps and AppRunner for more info.
In our current form we are relying solely on the CouchDB service from Cloudant for our data layer. This single point of failure leaves us vulnerable, so the near term goal is to run a CouchDB instance on the Webserver, and back it up in real time by doing CouchDB database replication to Cloudant. This setup has the added benefit of allowing us to use the databases on Cloudant as warm failover options with some simple failure detection logic in Apprunner.
Our short term goal is to expose additional database engines like MySQL and Redis to Enginemill applications through the Apprunner sandbox on the Webserver. We will do this only on an as needed basis. The challenge is providing close to real time replication for each of them.
Our medium term goal is to create a unified API to a general purpose data storage and query engine which is independent of the underlying implementations. This API would also be exposed to Enginemill applications through the Apprunner sandbox.
Maestro is the HTTP proxy and application controller which lives on the Webserver. Each web application bundle (Apprunner + Enginemill + application code) is running in an isolated process and using an open port on localhost to serve HTTP requests.
Maestro's main task is to accept application restart notifications via the DNode protocol. Currently these notifications are sent from the Enginemill SDK on a developer workstation whenever the developer deploys new application code. Each restart request is basically a remote procedure call with two parameters: the unique name of the web application on the webserver and the hostname the application accepts requests for.
Whenever Maestro receives a restart notification it looks for an open port on localhost and starts an Apprunner process on that port for the named web application. After the Apprunner process has successfully started, Maestro begins proxying HTTP requests for the named web application hostname to the new port on localhost. When this transition process is complete, Maestro shuts down the previous Apprunner process for that web application.
See Apprunner, Maestro, and Enginemill for more info.
Web applications are deployed directly from the Enginemill SDK on developer workstations. The deployment is made directly to the Webserver from the SDK through SSH using the rsync protocol. This requires the application developer to have the required SSH key to gain access to the EC2 instance, which is not ideal for security reasons.
See Enginemill for more info.