Use images for all services

[lognaturel]

Goal:

• Typical users never need to rebuild frontend and backend so their upgrades and server administration are faster

Nice to have:

• Releases are slightly more automated, reducing overhead and the risk of errors

Requirements:

• There's still an easy path (maybe just in docs) for advanced users to customize frontend/backend/ops code if needed
• The staging server continues to be updateddaily to match frontend, backend, ops
• There's still an easy path to deploying specific backend/frontend commits to the dev server

#676 builds nginx and service images. The enketo and secrets services can use stock base images instead.

Thinking doc (requires auth)

[lognaturel]

I gave some thought to the enketo and secrets images and I'd rather leave them as-is.

For secrets, we could remove secrets.dockerfile and change the Docker compose block to something like:

  secrets:
    image: node:20.12.2-slim
    volumes:
      - ./files/enketo/generate-secrets.sh:./generate-secrets.sh:ro
      - secrets:/etc/secrets
    command: './generate-secrets.sh'

We could do something similar for Enketo (use volumes instead of COPY directives in the dockerfile, specify the script from docker compose) but I don't see much advantage.

For someone who wants to do their own orchestration, it makes no real difference as far as I can tell. They'll use the base image and then decide how they want to define the secrets volume generate and manage their secrets, get the config files in the right place, and run the config and launch scripts.

The tradeoff that I see is between making the base versions really clear in the docker compose file vs. keeping everything about a service more contained in a Dockerfile. I have a slight preference for the latter (status quo).

@spwoodcock @tobiasmcnulty does that sound reasonable? Anything I might be missing?

[tobiasmcnulty]

It's nice from my perspective to have all the configuration and start-up files needed by Enketo bundled nicely into its own image. Furthermore, as long as the Central install requires a package to be installed that's not in the base image, I don't think there's any way to get around it.

[lognaturel]

the Central install requires a package to be installed that's not in the base image

I haven't tried this yet but I believe it should be possible to do an apt-get call in a shell script if needed.

The only part that's not in the Enketo base image is the config templating. The reason it's not in there is to give flexibility as to how that's done based on the deployment strategy. For example, the default Central install uses a shared volume to generate and persist secrets but in a specific deployment context it might make more sense to generate the secrets and then persist them in whatever deployment platform's secrets repository and template the configs from there.

[tobiasmcnulty]

Thanks for the quick replies!

I haven't tried this yet but I believe it should be possible to do an apt-get call in a shell script if needed.

Maybe I misunderstand, but in case this is the thought...please don't install packages during the container startup. Apt could fail or parts of the internet could be inaccessible (a real issue in our use case!), slowing or preventing the container launch. Even for people with fast and reliable connections, it leads to a (slightly) slower start, and confusing behavior when the container is executed with some other startup command and a package is missing.

The only part that's not in the Enketo base image is the config templating. The reason it's not in there is to give flexibility as to how that's done based on the deployment strategy. For example, the default Central install uses a shared volume to generate and persist secrets but in a specific deployment context it might make more sense to generate the secrets and then persist them in whatever deployment platform's secrets repository and template the configs from there.

Honestly I feel like what you have already is great. Startup scripts that modernize legacy applications by reading configuration from the environment provide real value. Sure, someone could sort out how to configure it by hand (and having these files in the image doesn't prevent that), but why bother when you've already done the hard part? 🙂

One nice-to-have might be to separate the startup script into its ENTRYPOINT and CMD components, so the command could be overridden while still getting the benefit of the configuration steps. But I can't say I've run into a need for that for this image yet, just raising it in case it seems useful!

[lognaturel]

please don't install packages during the container startup

Ah yes, that makes sense. It was too half baked of an idea!

We maintain the Enketo image so maybe we can just have it install gettext and then putting the files in the right place and doing the templating can be in the docker compose file. How does that sound?

[spwoodcock]

Sounds good to me 👍

Another option could be to add an additional container to the stack that has the gettext dependency, mount the required files, and run the command on start. The enketo container could have a depends_on service_completed_successfully before it starts and mounts the same files. The additional container would shut down after.

Not ideal, but possible if the source enketo image cannot be modified!
It adding the extra dependency isn't a big deal, then that's definitely the way to go though 😄

[tobiasmcnulty]

We maintain the Enketo image so maybe we can just have it install gettext and then putting the files in the right place and doing the templating can be in the docker compose file. How does that sound?

Oh, I missed this, and I can see why you might prefer to maintain only one Enketo image! Adding at least the gettext package to the source image sounds good to me. It'd be nice if it came with an entrypoint to create the configuration file based on the environment, but I can see how you might want to keep that in this repo, too.

If the configuration files are not in the image, we would need to mirror them in our helm chart. But I believe that's easily scripted and a small price to pay if we can run an official image directly.

[lognaturel]

Possibly of interest: Enketo can be configured directly using environment variables. That might be most appropriate for your context.

It'd be nice if it came with an entrypoint to create the configuration file based on the environment, but I can see how you might want to keep that in this repo, too.

The primary reason we took that work out of the Enketo image is because it felt like it overlapped with the direct configuration through env variables and because there's variation in what's defaulted vs. exposed for configuration across Enketo consumers. Central doesn't use the environment var based configuration because we wanted to specify a mix of defaults and templated configs and it felt nice to do that all in an Enketo-specific file. Do those seem like reasonable reasons? We can certainly revsit if not!

If the configuration files are not in the image

And here you mean the Central templated Enketo config, right? There is a base config with default values there but indeed the templated one is something Central-specific. Again, the reasoning is that for stock Central installs we have certain things we want to set as defaults and others we want to explicitly make configurable.