DNS not working on macOS

[grahamb]

I'm trying to transition from Dinghy to Docker for Mac & Dory, but I'm having an issue with DNS not resolving.

macOS: 10.14.4
Docker: 18.09.3, build 774a1f4
Ruby: 2.3.7p456
Dory: 1.0.3

Dory reports that the DNS service is running, and I can see something listening on 127.0.0.1:53 on my Mac. However, I can't resolve any *.docker hostnames. For example, running a plain nginx container, I can access it on localhost, but not on a .docker hostname:

$ docker run -d -e VIRTUAL_HOST=nginx.docker -p 8080:80 nginx:latest

$ curl http://localhost:8080 
... returns nginx page ...

$ curl http://nginx.docker
... hangs ...

$ dig @127.0.0.1 nginx.docker

; <<>> DiG 9.10.6 <<>> @127.0.0.1 nginx.docker
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

$ dig @$(dory ip dns) nginx.docker 
; <<>> DiG 9.10.6 <<>> @172.17.0.3 nginx.docker
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

$ cat /etc/resolver/docker
# added by dory
nameserver 127.0.0.1
port 53

I installed tcpdump on the dory_dnsmasq container, and I can see it receiving UDP packets when I try to do a dig @127.0.0.1 nginx.docker:

tcpdump -i eth0 udp port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:41:34.024636 IP 172.17.0.1.54947 > 437d613413fa.53: 43917+ [1au] A? nginx.docker. (41)
20:41:34.024751 IP 437d613413fa.53 > 172.17.0.1.54947: 43917*$ 1/0/0 A 127.0.0.1 (46)
20:41:34.024868 IP 437d613413fa.42445 > 192.168.65.1.53: 63392+ PTR? 1.0.17.172.in-addr.arpa. (41)
20:41:34.025668 IP 192.168.65.1.53 > 437d613413fa.42445: 63392 NXDomain 0/0/0 (41)
20:41:34.026201 IP 437d613413fa.41651 > 192.168.65.1.53: 36631+ PTR? 1.65.168.192.in-addr.arpa. (43)
20:41:34.026934 IP 192.168.65.1.53 > 437d613413fa.41651: 36631 NXDomain 0/0/0 (43)

20:41:39.029900 IP 172.17.0.1.54947 > 437d613413fa.53: 43917+ [1au] A? nginx.docker. (41)
20:41:39.030022 IP 437d613413fa.53 > 172.17.0.1.54947: 43917*$ 1/0/0 A 127.0.0.1 (46)
20:41:44.035306 IP 172.17.0.1.54947 > 437d613413fa.53: 43917+ [1au] A? nginx.docker. (41)
20:41:44.035476 IP 437d613413fa.53 > 172.17.0.1.54947: 43917*$ 1/0/0 A 127.0.0.1 (46)
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel

scutil --dns shows the resolvers for .docker and .dev:

$ scutil --dns
DNS configuration

resolver #1
  search domain[0] : my.dhcp.domain
  nameserver[0] : xxx.xxx.xxx.xxx
  nameserver[1] : yyy.yyy.yyy.yyy
  if_index : 8 (en0)
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

resolver #8
  domain   : docker
  nameserver[0] : 127.0.0.1
  port     : 53
  flags    : Request A records, Request AAAA records
  reach    : 0x00030002 (Reachable,Local Address,Directly Reachable Address)

resolver #9
  domain   : dev
  nameserver[0] : 127.0.0.1
  port     : 53
  flags    : Request A records, Request AAAA records
  reach    : 0x00030002 (Reachable,Local Address,Directly Reachable Address)

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : my.dhcp.domain
  nameserver[0] : xxx.xxx.xxx.xxx
  nameserver[1] : yyy.yyy.yyy.yyy
  if_index : 8 (en0)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  search domain[0] : my.dhcp.domain
  nameserver[0] : xxx.xxx.xxx.xxx
  nameserver[1] : yyy.yyy.yyy.yyy
  if_index : 5 (en1)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

[FreedomBen]

Nice troubleshooting @grahamb ! Sorry I'm not sure what's going on. It might be an upstream issue in the proxy (Dory is probably behind dinghy; it's been a little while since we've updated). You could try pulling the latest release of dinghy-http-proxy by adding an image key to the settings for nginx_proxy in your dory config file (~/.dory.yml is the default location). That will tell dory to use whatever custom image you specify. It's currently at codekitchen/dinghy-http-proxy:2.5.10 but the latest is codekitchen/dinghy-http-proxy:2.6.1

So in the example config, you'd set:

---
dory:
  # Be careful if you change the settings of some of
  # these services.  They may not talk to each other
  # if you change IP Addresses.
  # For example, resolv expects a nameserver listening at
  # the specified address.  dnsmasq normally does this,
  # but if you disable dnsmasq, it
  # will make your system look for a name server that
  # doesn't exist.
  dnsmasq:
    enabled: true
    domains:               # array of domains that will be resolved to the specified address
      - domain: docker     # you can set '#' for a wilcard
        address: 127.0.0.1 # return for queries against the domain
      - domain: dev
        address: 127.0.0.1
    container_name: dory_dnsmasq
    port: 53  # port to listen for dns requests on.  must be 53 on linux. can be anything that's open on macos
    # kill_others: kill processes bound to the port we need (see previous setting 'port')
    #   Possible values:
    #     ask (prompt about killing each time. User can accept/reject)
    #     yes|true (go aheand and kill without asking)
    #     no|false (don't kill, and don't even ask)
    kill_others: ask
    service_start_delay: 5  # seconds to wait after restarting systemd services
  nginx_proxy:
    enabled: true
    container_name: dory_dinghy_http_proxy
    https_enabled: true
    ssl_certs_dir: ''  # leave as empty string to use default certs
    image: codekitchen/dinghy-http-proxy:2.6.1
  resolv:
    enabled: true
    nameserver: 127.0.0.1
    port: 53  # port where the nameserver listens. On linux it must be 53
      

Please let me know if that doesn't make sense. If you try it and it works, I can release an update to dory bumping the proxy version number.

[subpardaemon]

any development in this? i tried the fix suggested by @FreedomBen but it's the same, i'm getting nxdomain results with name servers seemingly timing out.

[tripox]

Still having this issue? @subpardaemon @grahamb
I see mine is using freedomben/dory-http-proxy:2.5.9.1 and I have no issues.

I also made dory installable with Brew: https://formulae.brew.sh/formula/dory#default

Latest version of Dory is 1.1.1