Support Connection Pooling

[eric-fong]

Could this library support Connection Pooling?

http://mongoc.org/libmongoc/current/connection-pooling.html#connection-pooling
http://mongoc.org/libmongoc/current/mongoc_client_pool_t.html

mongoc_client_pool_t
mongoc_client_pool_pop
mongoc_client_pool_push
mongoc_client_pool_set_error_api

The following method may be also needed for gc?
mongoc_client_pool_destroy (pool);
mongoc_uri_destroy (uri);

mongoc_cleanup ();

Here is the other mongoc binding driver in ffi, for your reference.
https://github.com/lloydzhou/luajit-mongo/blob/master/mongo.lua

---

Thanks.

[neoxic]

Thanks for your feature request!

I guess it's worth thinking about. The only issue would be that it's not possible to call mongoc_cleanup() reliably from Lua.

It's indeed possible to call some sort of a "library destructor", but it would be incorrect to make a call to mongoc_cleanup() in it because there's no guarantee that some other Lua instance (or even a totally unrelated instance) in a separate thread is not using Mongo C Driver. To the contrary, multiple calls to mongoc_init() are perfectly fine.

Without a call to mongoc_cleanup(), a process will be left with some sort of a worker thread running in the background. Tools like Valgrind will show leaks for such a process at the very least. There may be other minor side effects I guess.

[eric-fong]

Thanks for your reply.
I would like to use client_pool in a nginx luajit module.
Currently, mongoc_cleanup() is not implemented, but the resource should be released when the worker thread exist. And so that client_pool should be released when worker exist, right?

It seem not a problem when using nginx luajit VM, am I correct?

[neoxic]

I assume you're talking about OpenResty.

First of all, there's not much sense in using Mongo C Driver (and lua-mongo as a result) in Nginx for one simple reason - it does not have an asynchronous API which means it will block a worker process on each I/O operation. In other words, the absence of an async API defies the whole purpose of Nginx's asynchronous I/O engine.

Not calling mongo_cleanup() doesn't seem to cause any real problems for most cases except when debugging is needed.

[psvz]

Hi @neoxic - my 2 cents...

Rest servers with db, whether nginx/mongo or in general, can't conceptually be asynchronous with respect to each other. That is because the server gets a request and sends a response to a client, and DB transactions must take place in between => unavoidably synchronously (if we not fancy thread-storms and OOM events but that's another story). However, optimization requested here has little to do with "mongo made asynchronous".

The only thread-safe model in mongoc driver is pooling clients and corresponding connections - see here. This thread-safe pool can be shared by coroutines, pertinent to OpenResty see here. That would allow multiple coroutines concurrently process Rest API requests (yes - waiting for meaningful DB transactions among other stuff) without wasting time on making and dropping server-DB connections.

Appreciate your thoughts on this.