MacOS: "no response" for all queries with VPN

[thomasmerz]

On MacOS in home office I'm getting no response for all queries. For example:

🍎 ✘ [58%] ⚡ 🌱🍃🐣🌸 Thomas.Merz@dm-C02CGH01MD6M:~/Downloads [ttys003/2865]
10:19 $ s dnspeep | grep outlook.office365.com
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
AAAA  outlook.office365.com          172.23.104.73        <no response>
A     outlook.office365.com          172.23.104.73        <no response>
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
^C

But both host and nslookup are responding with an IP adress:

🍎 ✘ [58%] ⚡ 🌱🍃🐣🌸 Thomas.Merz@dm-C02CGH01MD6M:~/Downloads [ttys003/2866]
10:20 $ host outlook.office365.com
outlook.office365.com is an alias for outlook.ha.office365.com.
outlook.ha.office365.com is an alias for outlook.ms-acdc.office.com.
outlook.ms-acdc.office.com is an alias for FRA-efz.ms-acdc.office.com.
FRA-efz.ms-acdc.office.com has address 52.97.201.114
FRA-efz.ms-acdc.office.com has address 52.97.179.194
FRA-efz.ms-acdc.office.com has address 40.101.121.2
FRA-efz.ms-acdc.office.com has IPv6 address 2603:1026:204::2
FRA-efz.ms-acdc.office.com has IPv6 address 2603:1026:207:14::2
FRA-efz.ms-acdc.office.com has IPv6 address 2603:1026:207:cd::2
🍏 ✔ [58%] ⚡ 🌱🍃🐣🌸 Thomas.Merz@dm-C02CGH01MD6M:~/Downloads [ttys003/2867]
10:20 $ nslookup outlook.office365.com
Server:         172.23.104.73
Address:        172.23.104.73#53

Non-authoritative answer:
outlook.office365.com   canonical name = outlook.ha.office365.com.
outlook.ha.office365.com        canonical name = outlook.ms-acdc.office.com.
outlook.ms-acdc.office.com      canonical name = FRA-efz.ms-acdc.office.com.
Name:   FRA-efz.ms-acdc.office.com
Address: 52.97.179.194
Name:   FRA-efz.ms-acdc.office.com
Address: 40.101.121.2
Name:   FRA-efz.ms-acdc.office.com
Address: 52.97.201.114

🍏 ✔ [58%] ⚡ 🌱🍃🐣🌸 Thomas.Merz@dm-C02CGH01MD6M:~/Downloads [ttys003/2868]
10:20 $

On my linux client (192.168.42.241) at home I see a good response:

A     outlook.office365.com          192.168.42.241       CNAME: outlook.ha.office365.com, A: 52.98.41.162, A: 52.98.66.210, A: 40.101.146.178, A: 52.98.37.98, A: 40.101.147.114, A: 52.98.89.34, A: 40.101.146.194, A: 52.98.82.210
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
Error parsing DNS packet: Failed to parse DNS packet: query type 65 is invalid
AAAA  outlook.office365.com          192.168.42.241       CNAME: outlook.ha.office365.com, AAAA: 2603:1046:404:15::2, AAAA: 2603:1046:c09:1003::2, AAAA: 2603:1046:c09:1802::2, AAAA: 2603:1046:404:a::2, AAAA: 2603:1046:403::2, AAAA: 2603:1046:c09:1120::2, AAAA: 2603:1046:402:1::2, AAAA: 2603:1046:c09:1804::2

172.23.104.73 is DNS resolver in my home office VPN (Cisco AnyConnect) for my MacOS.

On a linux server at my work I also see a good response with 172.23.104.73 as DNS resolver:

🐧  admin-f6l@tst-thomas-83.lxdev:~$ s ./dnspeep
query name                           server IP            response
A     lxos-monitoring-probe.dm-drogeriemarkt.com 127.0.0.1            A: 172.23.75.243
A     web.de                         172.23.104.73        A: 82.165.230.17, A: 82.165.229.138
AAAA  web.de                         172.23.104.73        NOERROR

Without VPN my local DNS resolvers 192.168.0.1 or 192.168.42.241 (in different home-Wifis) are used on MacOS and responses are good:

CNAME: outlook.office365.com, CNAME: outlook.ha.office365.com, CNAME: outlook.ms-acdc.office.com, CNAME: FRA-efz.ms-acdc.office.com, A: 52.97.135.114, A: 40.101.121.2, A: 40.101.12.34

❓ Is this a general problem related to VPNs?

[badnetmask]

238: //if we don't see a response to a query within 1 second, print "<no response>"

This is just your DNS server, while on VPN, taking longer than 1 second to respond. You can probably confirm that if you use the "time" command before your "nslookup".

That's indeed something that could be improved: I'm not sure how it works in Mac, but in Linux the dnspeep app could just use glibc's timeout instead of a hardcoded timeout.

[jvns]

Can you share a pcap file of DNS packets from your Mac? (where it's not working). You can get one with sudo tcpdump -i any port 53 -c 100 -w dns_packets.pcap. It'll be easiest for me to reproduce and fix the issue with a pcap file.

You can email it to julia@jvns.ca if you don't want to post it to github.

[thomasmerz]

@badnetmask - Well, it's not taking so long to respond. One second would be very, very intolerable to deal with… I'm used up to 10-20 ms.

17:20 $ sudo dnspeep | awk '{now=strftime("%Y-%m-%d %T "); print now $0}'
2021-04-01 17:21:01 query name                           server IP            response
2021-04-01 17:21:05 AAAA  winatp-gw-weu.microsoft.com    172.23.104.73        <no response>
2021-04-01 17:21:05 AAAA  wdatp-prd-weu-3.westeurope.cloudapp.azure.com 172.23.104.73        <no response>
2021-04-01 17:21:05 A     wdatp-prd-weu-3.westeurope.cloudapp.azure.com 172.23.104.73        <no response>
2021-04-01 17:21:05 A     winatp-gw-weu.microsoft.com    172.23.104.73        <no response>
…
🍏 ✔ [66%] ▅ 🌱🍃🐣🌸 Thomas.Merz@dm-C02CGH01MD6M:~ [ttys004/3051]
17:21 $ time nslookup winatp-gw-weu.microsoft.com
Server:		172.23.104.73
Address:	172.23.104.73#53

Non-authoritative answer:
winatp-gw-weu.microsoft.com	canonical name = sevillecloudgateway-weu-prd.trafficmanager.net.
sevillecloudgateway-weu-prd.trafficmanager.net	canonical name = wdatpprd-weu.securitycenter.windows.com.
wdatpprd-weu.securitycenter.windows.com	canonical name = k8stm-prd-weu.trafficmanager.net.
k8stm-prd-weu.trafficmanager.net	canonical name = wdatp-prd-weu-3.westeurope.cloudapp.azure.com.
Name:	wdatp-prd-weu-3.westeurope.cloudapp.azure.com
Address: 51.137.31.2

**real	0m0.073s**
user	0m0.002s
sys	0m0.019s

[thomasmerz]

@jvns - with wireshark I can see in the pcap file that there's a response:

Sending pcap file directly to you, because I'm not sure if it may contain some stuff, that my company don't want to see on public pages like GitHub.

[badnetmask]

Just as additional information, in case it helps: I have run dnspeep while connected to my company VPN for the entire day, and didn't get a single "no response" message.

[jvns]

I'm still not sure what's going on here even with the pcap file you sent but I made a new release (0.1.1) that fixes a different bug -- can you try it and see if it helps?

[thomasmerz]

Also with latest 0.1.1. as soon as I turn on my VPN all queries have "no response"… 😢 And when VPN is off almost all queries have a response again (there are still some with "no response", but quite a very few only - which also happens sometimes on different Linux distributions).

[badnetmask]

I don't mean to sound pushy, but I think having the response time would help debug this problem.

[thomasmerz]

timestamp      query name                           server IP            elapsed   response
Error parsing DNS packet: Failed to parse DNS packet: Could not decode QType: 65
Error parsing DNS packet: Failed to parse DNS packet: Could not decode QType: 65
14:32:03.885   A     outlook.ha.office365.com.      172.23.104.73                  <no response>
14:32:03.885   AAAA  outlook.ha.office365.com.      172.23.104.73                  <no response>
14:32:03.913   A     fra-efz.ms-acdc.office.com.    172.23.104.73                  <no response>
14:32:03.913   AAAA  fra-efz.ms-acdc.office.com.    172.23.104.73                  <no response>
Error parsing DNS packet: Failed to parse DNS packet: Could not decode QType: 65
Error parsing DNS packet: Failed to parse DNS packet: Could not decode QType: 65
14:32:09.411   AAAA  lh3.ggpht.com.                 172.23.104.73                  <no response>
14:32:09.411   A     lh3.ggpht.com.                 172.23.104.73                  <no response>
14:32:09.439   AAAA  photos-ugc.l.googleusercontent.com. 172.23.104.73                  <no response>
14:32:13.198   A     sso.redhat.com.                172.23.104.73                  <no response>
14:32:13.198   AAAA  sso.redhat.com.                172.23.104.73                  <no response>
14:32:13.255   A     e9961.x.akamaiedge.net.        172.23.104.73                  <no response>
14:32:13.255   AAAA  e9961.x.akamaiedge.net.        172.23.104.73                  <no response>
14:32:21.608   A     lxmonitoring.dm-drogeriemarkt.com. 172.23.104.73                  <no response>
14:32:21.608   AAAA  lxmonitoring.dm-drogeriemarkt.com. 172.23.104.73                  <no response>
14:32:21.644   AAAA  lxmonitoring.lxprod.ka.de.dm-drogeriemarkt.com. 172.23.104.73                  <no response>
14:32:23.988   A     safebrowsing.googleapis.com.   172.23.104.73                  <no response>
14:32:23.988   AAAA  safebrowsing.googleapis.com.   172.23.104.73                  <no response>

As already timestamped by piping to awk and now with -t paramter (Release 0.1.2): How does this help in this case? 🤷🏻‍♂️ Response is much faster than one second which also tcpdump/wireshark already has proven that there's a valid/good response - but dnspeep doesn't get it right with (my?) VPN activated.

[thomasmerz]

Hi @jvns , do you have any idea regarding this issue?

With the latest release of dnspeep this is still an issue on my Macbook when Cisco VPN is activated (secured: private IP-networks and my company's networks, non-secured: 0.0.0.0/0; 172.23.104.73 is company's DNS resolver):

$ sudo dnspeep
query name                           server IP            response
A     winatp-gw-weu.microsoft.com.   172.23.104.73        <no response>
AAAA  winatp-gw-weu.microsoft.com.   172.23.104.73        <no response>
AAAA  wdatp-prd-weu-6.westeurope.cloudapp.azure.com. 172.23.104.73        <no response>
…

But there's no such issue with Wireguard VPN activated (AllowedIPs = ::/0, 1.0.0.0/8, 2.0.0.0/8, 3.0.0.0/8, 4.0.0.0/6, 8.0.0.0/7, 11.0.0.0/8, 12.0.0.0/6, 16.0.0.0/4, 32.0.0.0/3, 64.0.0.0/2, 128.0.0.0/3, 160.0.0.0/5, 168.0.0.0/6, 172.0.0.0/12, 172.32.0.0/11, 172.64.0.0/10, 172.128.0.0/9, 173.0.0.0/8, 174.0.0.0/7, 176.0.0.0/4, 192.0.0.0/9, 192.128.0.0/11, 192.160.0.0/13, 192.169.0.0/16, 192.170.0.0/15, 192.172.0.0/14, 192.176.0.0/12, 192.192.0.0/10, 193.0.0.0/8, 194.0.0.0/7, 196.0.0.0/6, 200.0.0.0/5, 208.0.0.0/4, 10.13.13.1/32; 10.13.13.1 is DNS-Resolver within Wireguard-VPN)… 🤔

$ sudo dnspeep
query name                           server IP            response
A     ipv4only.arpa.                 10.13.13.1           A: 192.0.0.171, A: 192.0.0.170
A     itunes.apple.com.              10.13.13.1           CNAME: itunes-cdn.itunes-apple.com.akadns.net., CNAME: itunes.apple.com.edgekey.net., CNAME: e673.dsce9.akamaiedge.net., A: 95.100.188.41
A     api.apple-cloudkit.com.        10.13.13.1           CNAME: api.apple-cloudkit.fe.apple-dns.net., A: 17.248.150.24, A: 17.248.150.48, A: 17.248.150.112, A: 17.248.150.43, A: 17.248.150.135, A: 17.248.150.77, A: 17.248.150.152, A: 17.248.150.16
HTTPS api.smoot.apple.com.           10.13.13.1           NOERROR
A     1-courier.sandbox.push.apple.com. 10.13.13.1           CNAME: 1.courier-sandbox-push-apple.com.akadns.net., CNAME: us-sandbox-courier-e4.push-apple.com.akadns.net., A: 17.57.146.4
A     apple.com.                     10.13.13.1           A: 17.253.144.10
A     detectportal.firefox.com.      10.13.13.1           CNAME: detectportal.prod.mozaws.net., CNAME: prod.detectportal.prod.cloudops.mozgcp.net., A: 34.107.221.82
A     1-courier.push.apple.com.      10.13.13.1           CNAME: 1.courier-push-apple.com.akadns.net., CNAME: eu-north-courier-4.push-apple.com.akadns.net., A: 17.57.146.136, A: 17.57.146.140, A: 17.57.146.142, A: 17.57.146.133, A: 17.57.146.141, A: 17.57.146.132
A     alive.github.com.              10.13.13.1           CNAME: live.github.com., A: 140.82.112.25
A     push.services.mozilla.com.     10.13.13.1           CNAME: autopush.prod.mozaws.net., A: 44.240.140.78
A     web.whatsapp.com.              10.13.13.1           CNAME: mmx-ds.cdn.whatsapp.net., A: 31.13.92.52
HTTPS outlook.office365.com.         10.13.13.1           CNAME: outlook-exo.trafficmanager.net.
…

Or should we close this because this doesn't seem to be related to dnspeep?

[jvns]

I don't know what's going on and I had the time to look into it more yet. I'm happy to leave it open though -- it does seem like it could be an issue with dnspeep.

[thomasmerz]

Ok, feel free to ping me, if you need some more input from me.