Conversation
Remove the rspamd network and the static ip's in the compose file, because they are not mandatory. Static ip in compose file is not a best practice. Instead use the container name in `RSPAMD_DNS_SERVERS` environment variable.
|
It looks like it breaks DNS resolution at Rspamd |
How do you tested this? Will check it too. |
Something like |
|
Seems to work. |
|
It won't work - it will use resolv.conf in this case and something strange, such as 8.8.8.8 as a resolver. |
|
In fact, it is a problem in Rspamd that it does not resolve DNS resolvers names using |
|
@vstakhov hi, @BenRoe this PR is wrong. Vsevolod is right, DNS in software configured always as IP. The way it configured is same as in mailcow, mailu, mailinthebox or anything similar. The only change possible to be done here os to set dns resolver not in conf of rspamd, but via docker compose, but! it will be ip too, not a container name ;) Even when we speak about stuff like k8s, when you configure upstream in nodelocaldns or core dns, you can't use internal svc names there and you forced to use ips. |
|
@dragoangel I mean it might be better to adress this issue in Rspamd, as we are expected to run in docker environment and lack of the ability to resolve nameserver addresses using system resolving API (gethostbyname) is not very good. |
@vstakhov I do not see anything bad in adding that ability but it will be useful not for all containerized setups, mostly it will do a job for compose in this particular case. In k8s you most likely will stick to nodelocaldns (and not touch rspamd dns at all - use ip provided to container by k8s) which speak to some upstream root resolver like unbound. And even if not - it will be definitely service with static IP. Also question is - what if (gethostbyname) will fail? How much retries we expect? What if response will change over time? How often we will query it's resolving or we will check it only at startup. What if none of mentioned dns names of resolvers will not work at start and/or at runtime, etc. Technically DNS resolvers in every system always something static and pointed by IP. |
|
Reason I saying all that because now it asked for small particular use case, but in half year there will be a guy who will create issue why his rspamd fail to resolve dns (failed startup or lost resolution in the middle of the working process) when he pointed his dns to domain name without real need :) not in Container env at all |
|
Hello, I don't use this project directly but I do know this topic fairly well :) Collapsed for brevityYou have a few options for adjusting DNS configuration for Docker Compose. Worth noting is Docker (CLI without Compose) has defaulted to it's own legacy bridge network with differences from "user-defined" networks that Docker Compose uses by default. This also affects DNS, where the user-defined networks leverage an embedded/internal DNS service from Docker at I have some detailed insights here along with advice on how to handle that if you need to give priority to your own DNS service instead, otherwise Docker's internal DNS will have priority at resolving anything that'd match it's own internal networks DNS entries, hostnames, network aliases, etc (That also means PTR records, which was an issue I encountered 😅). If you'd like to modify If you give your DNS server priority but still want to leverage the convenience of Docker Compose DNS for connecting services, you can do as I demonstrated with CoreDNS (I've not used Unbound) and add a fallback to forward requests to services:
rspamd:
hostname: rspamd.example.internal
redis:
hostname: redis.example.internal
unbound:
hostname: dns.example.internalAlternatively you could avoid the You then need # Only relevant when using Docker with user-defined networks (Default network type for Docker Compose)
# Modify the file to swap the Docker DNS (127.0.0.11) to resolve your own preferred DNS server (172.20.42.10)
export WITH_NAMESERVER=172.20.42.10
sed "s/127.0.0.11/$${WITH_NAMESERVER}/" /etc/resolv.conf | sponge /etc/resolv.conf
In the past the daemon DNS fallback has been fixed, but it should be flexible to change that now, the user/admin must handle that on their system however. Same settings as what Docker Compose should be using by default AFAIK, other than user-defined networks having that extra I believe these days rather than
This should work AFAIK if Rspamd is fully in control of DNS queries from within the image, otherwise you'll get mixed behaviour. The DNS name can be queried to If you weren't using Docker Compose (eg:
This would be skipping the internal DNS obviously, so any private DNS records aren't going to be available. Even with my suggested modification of
Oh, I think I misunderstood from earlier then if you've got a different fallback for what DNS service is called. Just be mindful that I've observed issues with
Should you encounter that issue, you may want to patch # Relevant when the container lacks an FQDN hostname and services try to use it.
# WARNING: With an FQDN set via `--hostname` rDNS on a public IP with PTR to the same FQDN
# will fail resolving the FQDN back to the public IP, instead resolving the private container IP.
#
# Prepend preferred FQDN for services to resolve as the hostname `--fqdn` / `--domain`,
# Rather than the kernel hostname of this UTS namespace:
# https://docs.docker.com/reference/cli/docker/container/run/#uts
# https://docs.docker.com/reference/compose-file/services/#uts
export WITH_HOSTNAME='hello.world.test'
sed -E "s|($(cat /proc/sys/kernel/hostname))|${WITH_HOSTNAME} \1|" /etc/hosts | sponge /etc/hosts
User doesn't have to always configure that though, such as DHCP handing over an IP for DNS to query IIRC? There's also mDNS deployments which can kinda be setup to resolve beyond a single unique DNS label (non-standard).
As described earlier, if you can make a DNS query that would use My personal preference is to configure DNS at the appropriate places, rarely is that for services. I wasn't even aware that Rspamd had support for that 😅 Rspamd shouldn't need to be too concerned about it, but you should be mindful of the caveats I've shared above. If the DNS queries Rspamd needs to perform should bypass the internal DNS from Docker, then it's simpler to configure this in Rspamd's config since it supports this? So long as nothing else in it's container is performing DNS requests where it'd be inconsistent. Unbound can then redirect requests as appropriate to |
This comment was marked as outdated.
This comment was marked as outdated.
Yes, and this have to work that way. If you have issues due to this - fix them in some other way, because such behavior is consistent for any dockized environment, docker, containerd, compose & k8s and there no need to reinvert the wheel and trying to workaround it. Your rspamd MUST be able to resolve internal docker names if he living inside of docker network - because rspamd not just standalone system that resolving public names, it system that can rely on another components in same docker environment and due to that it must resolve their names... @polarathene I don't want to be or sound offensive but you creating own monster docker-antipattern mail system by shipping everything in one container which totally destroying idea of containerization. You need to check best practices and other projects that - none of have issues you describe because they build in correct way. |
What a horrible idea, sorry 🙈, no. @vstakhov I propose close this PR, it goes to nowhere. |
|
Didn't know that the topic is that complicated. |
Collapsed for brevity
I was simply sharing for the benefit of anyone here not aware of the various niche bits of knowledge on the topic, and valid ways to fix those should they be issues. Sometimes you may delegate containers DNS to another resolver, and you may have the expectation that you'd have the ability to supply records that would be used, not intercepted by Docker first, all I suggested was inversion. I don't even know why you're claiming that this is consistent with such environments when it's not. I already explained that As a result, no you don't have the same network consistency with containers across environments. k8s won't even let you set a hostname to a container, thus running software I cited is susceptible to the issue I mentioned when an FQDN was expected to be returned.
Where am I encouraging you to remove that ability? I simply said having Unbound route queries to Modifying Postfix can also be configured with a security restriction that connecting clients are required to provide FQDNs for their HELO. In the case of What do you propose to do in situations like that? The solutions I've described aren't bad when you understand the problems and what you're doing. Obviously if you don't have these problems in the first place with your software/deployment, then it's not something you should do, especially when there's better alternatives available (in the case of Another k8s specific issue I've had to accommodate support for is with preserving the original client IP from ingress/gateway. My experience with k8s is limited, but from what I was informed, we had to support external traffic with Proxy Protocol which the ingress controller such as Traefik would be configured for, but internal traffic within a pod containers would connect directly, with no ability to route through Traefik to append the expected ProxyProtocol headers, thus services such as Postfix and Dovecot needed to offer variations of their ports with ProxyProtocol enabled/disabled, such that both internal+external traffic was able to connect successfully. That wasn't an issue with Docker Compose, where I could easily control the internal DNS via a separate CoreDNS container. Another gripe I've had with k8s was lack of control over ulimits (but my upstream fixes to both Docker and containerd has resolved my issue with that now that containerd 2.0+ is seeing broader adoption). Then another caveat with Docker until recently was it's default enabled That IPv6 => IPv4 caveat was a bit of a concern for the ProxyProtocol support when it comes to Postfix and Dovecot, should anyone configure a wider subnet instead of only the specific trusted IPs of containers where ProxyProtocol headers are expected, or those that choose to relax security checks on private subnets for convenience of containers sending mail to bypass some DNS checks, etc.
You're nitpicking on one project I contribute to 🤷♂️ I have cleaned up CoreDNS DMS bundling multiple services into a single image is not an anti-pattern. That's commonly misunderstand and parroted. There's nothing wrong with multi-process images, a main process may spawn multiple child processes under the hood and it would have very little relevance when that's a single service right? Your concern is more about the use of a service manager as the main process managing other processes that represent distinct services. However the advice you're actually trying to point out is not recklessly bundle disjoint services into monolith images, which I would 100% agree with you on. DMS represents a logical service (a mail-server). It's a high-level component comprised of smaller individual services that are integrated to run within the single container. That has it's pros and cons, but as you can see from it's stars, it's one of the most popular containerized mail projects out there for that convenience. Very little is gained from splitting it apart into individual images, for which other competitors already offer (again with pros/cons). We do bundle Rspamd (which sadly we can't pin the version of in our builds because upstream insists against this, despite my offer to assist in adding that support in an easy to maintain manner that bring no real added burden to the project, but ok 🤷♂️), and while we also support running Redis within the image (and volume mounting for persistence), we also support using external container for Redis. If we were to offer a separate image for rspamd, we then have to add unnecessary complexity to support the configuration changes that DMS provides for the benefit of the user. There are various things I would change in the project if I could, but I'm not the only maintainer there and certain preferences I have contrast to the opinion of other maintainers, where none of us are paid to work on the project (none of my OSS work is financially supported, it's all voluntary), my goals there are only in supporting the users and making improvements when I can afford the time. The fact that I can't talk sense into upstream projects at times adds friction. I had no involvement in deciding to bring in rspamd, but I do my best to support it within DMS and to the users that need assistance with it, and as a result I've sometimes engaged with the project with only good intentions that tend to get dismissed, usually without valid reasoning (but I rather remain on good terms, so I try to respect/accept such decisions). So please don't judge my input from bias with my involvement on DMS just because it's one of those rare images that does things differently. My experience and contributions are not solely tied to that project, nor is there anything inherently wrong about it's approach to provide a single logical service.
I think I've provided plenty of insights already where DMS is irrelevant in this context and you can still hit problems (regardless of Docker or k8s). I'm aware of best practices, but I'm also experienced enough to know better where appropriate.
I've seen plenty of projects with hiccups when they tack on container support. You can still mess things up regardless (take a look at Deno's official images where their Alpine image copies their binaries to the Alpine base, but as it was built with glibc, they copy that over and use If you want to downplay my expertise and willingness to try assist by clarifying the cause of earlier observations conflicting, where I share knowledge to save the time of others having to figure such out as they're not always easy to learn about/discover, that's on you :\ |
you still can configure rspamd directly use correct dns without any interception by settings dns in options, I'm not understand what is stopping you? About other issues you describe - this is quite offtopic, and PR's is not a place for a broad chat from my view. But if try answer shortly: proxy proto is common way to provide srcip, yes - backend app should have dedicated port with proxy protocol and without proxy protocol, this is must have when you running any app behind L4 TCP proxy and you need get origin IP. About IPv6 traffic reaching IPv4 only container - unfortunately do not have IPv6 in k8s, but |
UPDATE: Actually the difference with ASN seems racey. I could repeat the command and What exactly is the not broken output meant to look like for reference? Because when you do it with the existing apparently working # NOTE: I added jq into the container:
$ rspamadm configdump --json | jq .options.dns
{
"timeout": 1,
"sockets": 16,
"retransmits": 5,
"nameserver": "round-robin:192.0.2.254:53"
}
$ rspamc -F foo@example.net -i 8.8.8.8 /dev/null
Results for file: /dev/null (1.54 seconds)
[Metric: default]
Action: reject
Spam: true
Score: 16.00 / 15.00
Symbol: COMPLETELY_EMPTY (15.00)
Symbol: R_SPF_FAIL (1.00)[-all:c]With the PR's proposed container name instead of IP for Rspamd DNS config (or also if you ignore this and just configure the container with the Compose $ rspamadm configdump --json | jq .options.dns
{
"timeout": 1,
"sockets": 16,
"retransmits": 5
}
$ rspamc -F foo@example.net -i 8.8.8.8 /dev/null
Results for file: /dev/null (3.21 seconds)
[Metric: default]
Action: reject
Spam: true
Score: 16.00 / 15.00
Symbol: ASN (0.00)[asn:15169, ipnet:8.8.8.0/24, country:US]
Symbol: COMPLETELY_EMPTY (15.00)
Symbol: R_SPF_FAIL (1.00)[-all]The only difference is the ASN symbol is present, which looking at the module docs doesn't seem like something to indicate that it's broken? You queried Initially I misunderstood from the IP choice and assumed
More context is needed for what the "correct" output is expected if it's wrong? |
I am aware of this. I thought I made that rather clear 🤷♂️ I was referring to if the apparent "breaks DNS resolution in Rspamd" statement was actually the case, that it coupled with that Unbound change to support
If you do not have any DNS issues caused by the internal The rest of my response is collapsed for brevity, expand if you care for more details:
Personal feedback
Most were about network caveats, or added context when responding to your decision to bring up my association to DMS and how you disapprove of the image not following "best practices" blindly. After responding to that, instead of acknowledging what I had said has some valid points, you'd rather dismiss it as off-topic (why bring it up in the first place?). Rather than a productive conversation on the issue at hand, you've been responding in a manner that only seems interested in criticizing my input along with 👎 reacts. Meanwhile anything I say that contradicts your criticisms is simply ignored. I recognize this type of discourse, and am better off avoiding engaging any further tbh as it does neither of us any good. I don't particularly appreciate feedback like "What a horrible idea, sorry 🙈, no." without any context as to why it's so horrible 😕 Maybe try some constructive feedback instead of just shutting me down? Response to answers
No separate ports are not mandatory. Only when services are implemented in a way that they lack the ability to handle both correctly on a single port. I've already had this argument with the HAProxy dev that authored the protocol, who became dismissive and quiet when I demonstrated proof of how it can be done securely despite his claims (since he had chimed in to discourage allowing such in another project (Caddy) on the basis of some vague security concern that when fleshed out was proven false). Feel free to dismiss me on that too. I have ample evidence on these topics available publicly, I am happy to be proven wrong with tangible facts but that's rare when I've invested enough time on a topic that I'm confident enough with reproductions to back it up.
Uhh yes... directly after calling out Rspamd's default configs I did bring up Postfix + Dovecot in the next paragraph. Considering I maintain DMS and I'm often involved in it's security those are things I try to be informed in. Well aware of the Open Relay risks, DMS doesn't have such wide trust by default, users would have to explicitly do that themselves.
If you disable the IPv6 connections wouldn't be routed, just refused instead. Regardless of Since Docker v27 I think IPv6 left experimental status and would be enabled by default if the host had it available, along with an IPv6 subnet pool, so shouldn't be an issue in modern environments. |
This comment was marked as outdated.
This comment was marked as outdated.
For obvious reasons - you need to have way to build image without compose?
I totally do not understand what you are trying to accomplish. You are fighting with shadows from my view. |
It's literally a Docker Compose example, if that's your argument you might as well tell them to build the image separately instead of using Compose Apart from the fact that the custom image build is pointless, it's only to bundle some config, something you can do just as easily with or without Compose via a bind volume mount.
That's ok, I could say the same to you as you're often not making sense when you dismiss anything I have to say, even when I provide information that invalidates your statements 🤦♂️ I'm still waiting on a reply for what possible use / value you're getting from forcing an implicit anonymous volume via It seems all you can do is 👎 me and are unwilling to appreciate / acknowledge anything I say that contradicts your own statements, even once it's evident how misinformed you are. What does that say about you? |
5 participants
Remove the rspamd network and the static ip's in the compose file, because they are not mandatory. Static ip in compose file is not a best practice.
Instead use the container name in
RSPAMD_DNS_SERVERSenvironment variable.