no connections for producer when using nsqlookupd

[jetspeed]

gem version 2.3.1

require 'nsq'

producer = Nsq::Producer.new(nsqlookupd: ['127.0.0.1:4161'], topic: 'qlc')
puts producer.connected?
producer.write('hello world')
producer.terminate

this cause:

gems/nsq-ruby-2.3.1/lib/nsq/producer.rb:92:in connection_for_write': No connections available (RuntimeError) from /home/was/.rvm/gems/ruby-2.2.2@425on222/gems/nsq-ruby-2.3.1/lib/nsq/producer.rb:65:in write_to_topic'
from /home/was/.rvm/gems/ruby-2.2.2@425on222/gems/nsq-ruby-2.3.1/lib/nsq/producer.rb:42:in write' from test.rb:6:in

'

[jetspeed]

this will occuer when using
ruby test.rb

but in irb, it gets connections.

why？ It seems the @connections not updated by the discovery Thread.

[bschwartz]

I'm guessing it's a race condition. If you're running it line by line in irb, it probably has a short time to do the discovery and make the connection. I'd try polling producer.connected? before writing. Maybe something like:

require 'nsq'
producer = Nsq::Producer.new(nsqlookupd: ['127.0.0.1:4161'], topic: 'qlc')
sleep(0.1) until producer.connected?
producer.write('hello world')
producer.terminate

[jetspeed]

you're right. could this be improved?

[bschwartz]

Yes, I see how this could be confusing :)

I suppose the most straightforward thing to do would be to block until a connection is made when creating the Producer, though you can still run into a case where all your connections drop if your nsqds go down.

I think this PR may be the better solution: #52. I haven't had time to fully review and incorporate it unfortunately.

[jetspeed]

Another problem, when pub to remote nsqds, some messages may loss,
If I add sleep(0.01) before write_to_topic in write method, nothing lost.

So I think the network latency will affect the message pub, and will cause message loss.

[Soulou]

Hi @jetspeed we had the same issue and it wasn't acceptable.

We've been using #52 in production since the PR has been done and fixed our message loss issue, our master branch is integrating the PR: https://github.com/Scalingo/nsq-ruby