Add spec for load balancing

[etlweather]

One thing that I see missing in those spec is defining a load balancing cluster. For example, I would want to do something like this:

{
    "virtualhost": "webapp1.domain.tld",
    "proxy": {
        "protocol": "balancer://",
        "hostname": "webapp1-cluster",
        "alias": "/",
        "backend_alias": "/"
    }
}
{
   "balancer": "webapp1-cluster",
   "members": [
               {
                "protocol": "http://",
                "hostname": "member1",
                "port": "8080",
                "route": "member1",
                "loadfactor": "50",
               }
              ]
}

In Apache HTTPD, this would translated into:

<VirtualHost *:80>
   ServerName webapp1.domain.tld

   ProxyPass / balancer://webapp1-cluster/ stickysession=JSSESSIONID|JSESSIONIDSSO
   ProxyPassReverse / balancer://webapp1-cluster/ stickysession=JSSESSIONID|JSESSIONIDSSO

   <Proxy balancer://webapp1-cluster >
      BalancerMember http://member1:8080
   </Proxy>
</VirtualHost>

I forked the httpd-cfg project - I am looking at spending some time to add the above and I think it could be a nice addition to your projects so I could send you a pull-request instead of just keeping it in my fork. But there would need to be a formalized spec I guess.

[jankaluza]

Thanks for the idea. I like it. I only think we have to support more configuration options. I've checked what httpd, nginx and haproxy supports (or need) and it seems we need following information to make the support fully configurable and usable across webservers:

• balancer name
• method for balancing (balancing_method): round-robin, least-connected (maybe also ip-hash?)
• method for session persistence (persistence):
  • method
    • generate_cookie - webserver generates the cookie to route requests to proper backends
    • use_cookie_or_url - webserver uses the cookie or URL id generated by the backend, first request is server to one of the backends
  • cookie_name
  • url_id
• balancer members
  • protocol
  • hostname
  • port
  • route
  • weight

I will have to do some tests with httpd and nginx to find out this is really enough for both to generate valid configuration. I'm not sure right now about the persistence methods. Maybe there will have to be more fields there, but that needs some testing.

Once I will do some tests, I will try to come up with some JSON specification.

Btw, I see you have put "balancer" and "virtualhost" definitions into separate JSON objects. Is that intentional? Are you thinking about using that in cloud when you would be spawning backend servers and the httpd-frontend would automatically add them?

I would ideally support something like that:

{
    "virtualhost": "webapp1.domain.tld",
    "proxy": {
        "protocol": "balancer://",
        "hostname": "webapp1-cluster",
        "alias": "/",
        "backend_alias": "/"
    },
    "balancers" {
        "webapp1-cluster": {
            "members": [
                {
                    "protocol": "http://",
                    "hostname": "member1",
                    "port": "8080",
                    "route": "member1",
                    "loadfactor": "50",
                }
            ]
        }
    }
}

And for the cloud use-case, the backend webconf could look like:

{
    "balancers" {
        "webapp1-cluster": {
            "members": [
                {
                    "protocol": "http://",
                    "hostname": "member1",
                    "port": "8080",
                    "route": "member1",
                    "loadfactor": "50",
                }
            ]
        }
    }
}

To make it consistent.

[etlweather]

Thanks. My example was only limited to the options I would need to serve things the way I have them now - the above could produce a valid configuration on httpd for what I need but yes, it wouldn't include all possible options someone else might need.

Having the balancer and virtualhost in separate object wasn't really intentional. It would have to be in the same VirtualHost configuration block on httpd to actually work (I think) as I don't think you can reference a balancer definition from one vhost in another.

But yes, I am thinking about using that in a cloud-like environment when I would be spawning backend servers and the httpd-frontend would automatically add them, though an internal private cloud.

Right now, I have an httpd server doing load balancing and routing to various backends. Everything is manually configured, backends are pretty fixed in VMs.

But I am working on switching to a Docker model where the Products Dev can just spawn containers easily to deploy new versions and pilot versions, etc. Thus I am trying to automate the thing so that they can start a Docker container with some labels describing what application, what sub-domain, what balancer group, what URL, etc should be proxied and the httpd service gets automatically configured and reloaded.

There are a few projects I saw that would be suited but they all lacked something or other. hipache was one example, but it only does subdomain routing. I can't specify URL routing.

I am also looking at producing a readable document that describes the current balancing and routing configuration so an admin can see it visually when troubleshooting (without having to open the configuration files). I.e. the output of the program would be a config file and a doc file.

I'll play around with your script from httpd-cfg. I am not sure that I totally understood all your other projects and how to use them (or if they are yet at a usable stage). Particularly, Configuring the web applications in Docker environment seems like what I am looking for but.... it was a bit sparse :)

[jankaluza]

The only thing missing for the pure Docker environment is "How to get the webconf-spec configuration files into the httpd container?".

I'm kind of focused on the Kubernetes or Openshift, where things are easier, because there's API server where I can store webconf-spec JSON and the httpd container can just download it using "curl" and reconfigure the httpd.

For the Docker, I need some shared place where the backends can put the configuration and from where the httpd (or another webserver) can download them. So far I have not decided what should be that place. I was thinking about volume shared between all the micro-webapps based containers, but since I'm using mainly Kubernetes, I was kind of waiting for someone who would like to use plain Docker and has some idea how it's supposed to work.

So, do you have some idea if the webconf-spec configuration should be stored in the shared docker volume, or it should be Docker image label and the main container would "somehow" access it? Is that possible with pure Docker?

[etlweather]

Got it! Well, I'll see if I can help with that. But I tried to get started yesterday and somehow the httpd-cfg program wants a "webapps" node but I couldn't find that in the doc (unless I'm blind and missed something obvious?).

[jankaluza]

It does not need "webapps", the httpd-cfg mentions webapps node, but it only uses it internally to store webapps. It uses that from historical reasons. I have removed that in the latest httpd-cfg commit.

You can check example webconf-spec configs in https://github.com/micro-webapps/webconf-spec-tests repository (in each directory, there's webconf.json file).

[jankaluza]

OK, I have cleaned up the webconf-spec a bit and also added load balancing support there. I will implement that in httpd-cfg on Monday and will investigate the plain Docker usage with micro-webapps. If you have some idea how you would like it to work, please share it :)

[etlweather]

Great! I tried some of the examples in the webconf-spec-tests project, but the error I get is:

$ ./httpd-cfg/httpd-cfg ./webconf-spec-tests/002/webconf.json /tmp/
Traceback (most recent call last):
  File "./httpd-cfg/httpd-cfg", line 643, in <module>
    sys.exit(main())
  File "./httpd-cfg/httpd-cfg", line 641, in main
    return generate_config(sys.argv[1], sys.argv[2])
  File "./httpd-cfg/httpd-cfg", line 618, in generate_config
    webapps_per_vhost, webapps_without_vhost = get_webapps_per_vhost(cfgs)
  File "./httpd-cfg/httpd-cfg", line 92, in get_webapps_per_vhost
    app_cfg["__name__"] = app
TypeError: 'unicode' object does not support item assignment

Not sure what I am doing wrong with this.

As for Docker, not sure how it all fits with your plans but what I was kind of looking at is that the user creates and runs containers with certain labels describing the webapp. (I am still learning all this so I don't know that it's all possible). In this case, the user is pushing various web application into production. There are always at least two container instances for one application to provide resiliency (different Docker hosts).

Similarly, when a new version of the webapp is being put on, the existing containers running version 1 remains active, new containers are started running version 2. The load balancer sends some traffic to version 1 and version 2 containers. When we're happy that the new version is running fine, we destroy the containers running version 1 and the load balancer stops sending traffic to those (they are removed from the load balancer group).

The webconf file would be derived from the data found in the labels of the running containers. Something like if the container is labeled as a 'webapp', then it's included. The labels would also tell me which load balancer group to put the container into, what virtual host, sub-domain, proxying path, etc.

How would this all happen dynamically... well I remember reading somewhere about Docker having some kind of way to do a webhook call or something on some events like starting a container. The idea then would be that some webhook program builds the json webconf file, runs the httpd-cfg program and reloads Apache. But I can't find that in the Docker documentation anymore.

I could do this through an Orchestration tool (e.g. Rundeck), i.e. the user does not log on to the Docker containers to start the new containers in the example above but run a pre-defined job and that job would take care of building the webconf file, reload Apache, etc.

I guess in some ways what I am trying to do is something Kubernetes does, but I felt going the whole Kubernetes route was too big of infrastructure for my needs... but maybe I should...

[jankaluza]

Will comment just the error right now (I will read and think about the rest of your comment later).

Try:

./httpd-cfg/httpd-cfg ./webconf-spec-tests/002/ /tmp/

It expects directory with configs as an input. I may add support for single file as an input too...

[etlweather]

Oh my. How did I miss that... it does say "input_directory_with_json_files"... Sorry!

[jankaluza]

Reading the rest of your comment, it really sounds like what Kubernetes does. That's also why micro-webapps lacks support for plain Docker containers right now... Everytime I was thinking about the use-case for web applications in plain Docker, it ended up with thoughts like "Everone will use something like Kubernetes for stuff like that".

That's why I was curious what your use-case with Docker is, because I personally use Kubernetes or Openshift for thing like that in my testing environment. Once you need to run things on multiple machines, I think plain Docker is not enough and the use-case on single machine is questionable (if you don't count testing applications).

[jankaluza]

I have just added support for "balancers" in the latest httpd-cfg commit. There are also two tests (021 and 022) in webconf-spec-tests repository.

[etlweather]

Thanks and I looked at it and it does look pretty awesome! I am currently off on a quick other project and then I'll be integrating this in my infrastructure! This looks great!

[jankaluza]

I've just wrote blog-post about load balancing using kubernetes and micro-webapps:

http://jkaluza.livejournal.com/6042.html