Using Node.js's built-in load-balancer in cluster mode for better distribution requests among workers

[firsturdiev]

bug description

Currently cobalt uses OS-level port-binding method for workers, in that way OS handles distribution of incoming requests among workers. But due to some OS-scheduling problems, it becomes very unbalanced, even node.js official warns about this:

But in another (default) method, node.js itself handles distribution in round-robin fashion with some minor latency in much reliable way.

The main reason of spawning multiple workers in cobalt is benefiting from equal distribution of requests so in this case using node.js built-in method rather than OS-level port binding looks right approach.

Open to discuss more about this by technical side

source

cobalt's code

[firsturdiev]

I'm in wait-mode of sending pull-request to fix this (just some lines of code), if repo owners interested in this approach of workers. I myself used this method for my own cobalt instance and results were significant

[dumbmoron]

results were significant

Do you have any benchmarks/stats? SO_REUSEPORT afaik should be very efficient, and does not suffer from what is described in OP (it should be actually more efficient than node:cluster balancing, because it does not rely on Node IPC).

[firsturdiev]

Do you have any benchmarks/stats?

Don't have benchmarks currently, but after reading about this on nodejs offical docs, I switched to built-in method, latency difference wasn't noticiable, and distibution was very equal (i checked manually with printing worker index next to each request log).

Will share with benchmark test video/proofs soon. Maybe you can also test it manually,

[firsturdiev]

Also one bottleneck about SO_REUSEPORT is that it uses it's own hashing mechanism (ip + client info) for distributing, not round-robin