Elasticsearch backend for logstash not being updated correctly

[andrew-wharton]

Looking at logstash/containerpilot.json...

The onStart.sh script, when called by the onChange handler for the elasticsearch backend in logstash/containerpilot.json, doesn't appear to update logstash.conf correctly.

There are 2 reasons for this.

1. I think the backend should be the elasticsearch-master service instead of the elasticsearch service.
2. Once the 'elasticsearch' string has been replaced in logstash.conf with an IP address, the replacement in onStart.sh script no longer works, since it looks for the 'elasticsearch' string which is no longer there.

[tgross]

I think the backend should be the elasticsearch-master service instead of the elasticsearch service.

The onStart.sh script seems to be correctly querying Consul for the elasticsearch-master but yes you're right that we should be watching changes in the master rather than the replicas. I think the current ES setup is avoiding that problem by having all the nodes by both master and data.

Once the 'elasticsearch' string has been replaced in logstash.conf with an IP address, the replacement in onStart.sh script no longer works, since it looks for the 'elasticsearch' string which is no longer there.

Yeah, we should probably replace it with a template file that gets copied to the location.

I'm currently hung up in ContainerPilot v3 work. I'd be happy to have a PR from you for these though.

[andrew-wharton]

No worries, I was thinking that this wouldn't be top of the priority list 😄. I'll have a look at it over the weekend or next week and see if I can put together something plausible.

As a side note, thanks for all the work you guys are doing on this autopilot pattern and the associated tooling. It's beautiful to see it in action when it's all set up and running smoothly!