Create 6.1-next variant of current Plone Images for incubating improvements

[fredvd]

From #147, where adding uv is suggested, but we're too late / close to the Plone 6.1 release.

Yes I realise this is rather/too late. But this tension os a consequence of keeping the main Plone version equal with the container image versions, having limited resources in the community, and looking at the ‘installer’ when the KGS of the sources are RC.

Maybe we can re-use the variant strategy used for frontend yarn/npm and the -bookworm update for the 6.0 plone-backend series with more recent OS and Python version.

We can add UV and other ‘professional experiments’ to a 6.1-next series of images where we can collaboratively work on improving the images in a public way. Then when Plone 6.2 is a about to be released, the ecosystem tested and refined in the 6.1-next images can be made the ‘stable’ main basis for the 6.2 images.

[fredvd]

Question/remark from @davisagli when I mentioned this: how are we going to add the variants, on a different branch, or in the workflows (as is done in plone-frontend).

If we add them to the workflows, the dockerfiles need to be duplicated in a seperate folder, the workflow files become larger, but you can release a new version with one version.txt. update.

With a separate branch (6.1.x-next) you need to update both the normal and -next branch and trigger the workflows.

My preference is I thiink the clarity of an as simple branch (no 'monorepo' subfolders) and github actions as possible, instead of duplicating the repo contents. One of the purposes for the 6.1-next configuration is to become the 6.2 'main' version. That would also be much easier with a branch per version-variant approach.

There will likely also be a 6.0-bookworm variant: ##163

[gforcada]

I've started to play with uv at work for Jenkins and Irma quite nice and simplifies the workflows quite a lot. If I have time during the sprint we could move our Jenkins jobs to it as well

[jensens]

Additional -next feature: Use cookiecutter-plone-instance in entrypoint to generate the configuration.
It supports all zope.conf features and most possibilities in zope.ini.

[davisagli]

@jensens You mean cookiecutter-zope-instance, right?

There is also @ericof's https://github.com/plone/plonectl/ which provides a nicer CLI for running Plone (but doesn't have a release or docs yet).

[jensens]

Yes, cookiecutter-zope-instance.

Cookiecutter is a tool we use already in our docs (cookieplone) and it is one tool for one purpose. And it is a simple configuration generator, nothing more.

cookiecutter-zope-instance

• does one thing, not more (UNIX philosophy) and thus is easy to maintain.
• is 100% documented with references to the original Zope documentation.
• is feature complete for zope.conf and ZMCL, zope.ini may need some tweaks (need use cases here).
• uses sane, namespaced configuration variables with names taken from the original Zope (or RelStorage, ....) documentation/variables where possible.
• also supports export/import/pack files for RelStorage
• works in Docker entry-points already providing a helper script with well named environment variables using a configurable prefix before the documented variable-name.
• plays nice together with other tools.
• is battle-tested in large, real life Volto and ClassicUI projects with Kubernetes and Docker Swarm.

Example usage in Docker

ENV COOKIECUTTER_ZOPE_INSTANCE_VERSION 2.1.1
ENV ZOPE_TEMPLATE /site/deployment/cookiecutter-zope-instance.zip
ENV ZOPE_CONFIGURATION_FILE /site/deployment/instance-from-environment.yaml
ENV ZOPE_BASE_FOLDER /instance

# presets for ZOPE
ENV INSTANCE_target /instance
ENV INSTANCE_debug_mode "false"

# ...

RUN \
  # ....
  # here pip, given a venv is there, could be uv tool install or prepare uvx, ... but you get the idea: it is preinstalled.
  pip install cookiecutter
  # ...
  # we download the cookiecutter-zope-instance local - since we do not want to download it in an entrypoint
  wget -O $ZOPE_TEMPLATE https://github.com/plone/cookiecutter-zope-instance/archive/refs/tags/${COOKIECUTTER_ZOPE_INSTANCE_VERSION}.zip &&\
  # we prepare the helper script to be ready
  wget -O deployment/transform_from_environment.py https://raw.githubusercontent.com/plone/cookiecutter-zope-instance/${COOKIECUTTER_ZOPE_INSTANCE_VERSION}/helpers/transform_from_environment.py &&\
  chmod u+x deployment/transform_from_environment.py &&\
  # ...

entrypoint

# ....
/venv/bin/python /site/deployment/transform_from_environment.py -o $ZOPE_CONFIGURATION_FILE
cookiecutter -f --no-input --config-file $ZOPE_CONFIGURATION_FILE --output-dir $ZOPE_BASE_FOLDER $ZOPE_TEMPLATE

Further, we have a Makefile with superior dependency resolution for all targets generated by mxmake (it works already in buildout.coredev) which can be easily extended with additional use-cases for one project (include.mk) or in general (add a template to mxmake itself, its open for contributions).

A full, standalone OCI-image example (not based on plone-backend, so it builds for ages and is not size optimized - about 590MB) based on the approach with cookiecutter-zope-instance and mxmake can be found here: https://github.com/bluedynamics/mximages-plone/tree/main/backend.

I prefer to not use a large, all-in-one, home grown Python scripts. Such is created quickly but we (not one person) need to maintain and document it all. plonectl could easily call cookiecutter-zope-instance instead of duplicating parts of it in there.

I have no idea why plonectl at all is needed. We have plonecli already which IMO would be the ideal candidate to extend if we want to go this path. IIRC this was an idea from the beginning. It is always easy to start greenfield, but I doubt it is sustainable.

[davisagli]

@jensens I've had good experiences with cookiecutter-zope-instance and have no objection to trying to use it here. You don't have to convince me about that one.

I have mixed feeling about make. It does a good job of managing dependencies between commands. But, some things I don't like:

• It's hard to get an overview of what commands are available without reading the Makefile
• When I do that, I find the syntax non-obvious
• Afaik It doesn't make it easy to pass additional args through to the commands

Also you say you prefer to not use a large, all-in-one, home grown script...well, sorry and maybe it's just my fault for not learning it properly yet, but that is what mxmake feels like to me.

The main point I wanted to make about plonectl is that I really like the UX it provides for its commands. It provides replacements for all the built-in Zope scripts but with better naming, help, and sane defaults. For example plonectl shell is like zconsole debug but it automatically calls setSite, sets up a request, etc. I don't care whether we use the specific implementation of Érico's plonectl prototype but we should copy its UX for what commands are available and how they work.

plonecli seems to have a bit different focus and doesn't look very relevant to my needs (at least as it exists currently). Its commands are focused more on creating project boilerplate and installing Plone (overlap with Cookieplone) and they still use buildout for that. There are commands for starting or debugging the instance but not for the other tasks involved in operating it.

Overall my personal "dream team" would be:

• make (perhaps mxmake) for dependency management (I'd rather have something in Python, but I can live with make)
• cookiecutter-zope-instance for creating instance configuration
• something hypothetical which provides a UX like plonectl but works with existing installed packages and an existing instance rather than trying to do that part itself (this could also possibly be an addition to Products.CMFPlone itself)

[jensens]

It's hard to get an overview of what commands are available without reading the Makefile

Actually last week I just created a PR for mxmake to add a help command showing everything available including description and parameters (as env vars). mxstack/mxmake#44

Indeed - the Makefile is like a bunch home grown short bash scripts, but with dependency management between them. In the end it is just a Makefile, and this is well established in the IT world. I tried a lot of other alternative task execution tooling with dependency management in the past, also possible Python alternatives to make. Nothing convinced me yet. But I am ready to learn.

I agree that we could provide a better tooling with Products.CMFPlone itself, first like small tools, i.e like a ploneconsole (or pconsole?) for a the zconsole debug with setSite et al. (do not call it shell, because from my understanding a shell is sh/bash/zsh/.. not a Python CLI, but well). One tool for one purpose at a time. Some plonectl then just integrates all of them. And if plonectl calls make too (or vice versa?) - why not?