add first feature metadata draft by evgeni · Pull Request #372 · theforeman/foremanctl

evgeni · 2026-02-12T14:25:46Z

No description provided.

docs/feature-metadata.md

evgeni · 2026-02-16T11:35:22Z

docs/feature-metadata.md

+
+Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).
+
+The metadata is a Hash with the feature name as the key and the feature definition as the value.


This file describes how the metadata looks like, but not where it's stored.

#309 stores it as a Hash in a Python file, but it's probably best suited as a standalone YAML file?

I agree -- a yaml file will be easier.

I agree that it needs to be a file. Do you think we need a single file per the whole foreman, or does it make more sense to have one per feature/plugin?

what would the benefit be if we have file-per-plugin? (I prefer single-file, unless there is a good reason not to)

Each plugin will own its metadata - it is easier to manage it in the plugin repo, than to know that a specific obscure file needs to be changed.

What if there are features that do not map to a plugin? Where would that metadata live?

right now it'd be a feature that has both foreman and foreman_proxy fields empty, but then something special in role

Each plugin will own its metadata - it is easier to manage it in the plugin repo, than to know that a specific obscure file needs to be changed.

How would the file from the repo come to Ansible?
How do people manage that today with Puppet?

ehelms · 2026-02-16T13:04:49Z

docs/feature-metadata.md

+    - foreman-tasks
+```
+
+The `foreman-tasks` feature is automatically enabled when the user requests Katello, thus also gaining the Hammer integration for tasks which would be missing if we'd only let the Ruby gem dependency pull in `foreman-tasks`.


To see if I understand everything correctly, if I combine the last two examples, what we would have is:

dynflow: internal: true foreman_proxy: dynflow foreman-tasks: foreman: foreman-tasks hammer: foreman_tasks dependencies: - dynflow katello: description: Katello foreman: katello dependencies: - foreman-tasks rh_cloud: description: Foreman RH Cloud foreman: foreman_rh_cloud requirements: - katello

Yeah. That would be the fuller example, I rather skipped multiple levels not to become too verbose (unless you think that's helpful?)

I needed to see this bigger, combined example to make sure I understood it will be one big hash with everything rather than a bunch of small hashes.

docs/feature-metadata.md

ehelms · 2026-02-16T13:06:40Z

docs/feature-metadata.md

@@ -0,0 +1,85 @@
+# Feature Metadata
+
+Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).


What makes a feature abstract?

to me mostly the fact that the implementation can be in foreman, in the proxy, in some sidecar (iop) or something unknown yet.

I would define a feature as a set of configuration steps. The configuration step could be "install gem", "install smart-proxy plugin", "install a container" or "modify a config file". I wonder if we can reuse Ansible native constructs (maybe a module) for such grouping instead of defining another grouping mechanism.

in reality, no "install" actions will happen tho, as we're working with pre-built containers.
possible actions (with todays architecture):

deploy a database

deploy a container

deploy a (set of) configuration

now that you mention that, it means that there is no such thing as smart-proxy machine anymore. It's just a container installed somewhere, and you have two actions to do: deploy the container on a desired machine and register a new smart proxy record in the Foreman database.
So we will need to orchestrate the container deploy on one machine and registering it on another.

That is where this is headed but we are still bound by the concepts of a foreman-proxy and one per machine (also think Capsules).

ShimShtein · 2026-02-16T13:58:36Z

docs/feature-metadata.md

@@ -0,0 +1,85 @@
+# Feature Metadata
+
+Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).


I would define a feature as a set of configuration steps. The configuration step could be "install gem", "install smart-proxy plugin", "install a container" or "modify a config file". I wonder if we can reuse Ansible native constructs (maybe a module) for such grouping instead of defining another grouping mechanism.

ShimShtein · 2026-02-16T14:01:19Z

docs/feature-metadata.md

+
+Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).
+
+The metadata is a Hash with the feature name as the key and the feature definition as the value.


I agree that it needs to be a file. Do you think we need a single file per the whole foreman, or does it make more sense to have one per feature/plugin?

ShimShtein · 2026-02-16T16:41:38Z

docs/feature-metadata.md

+The following properties are defined:
+* `description` (_String_): A human-readable description of the feature, can be used in documentation/help output.
+* `internal` (_Boolean_): Whether the feature is user visible (shows up in documentation/help) or internal (just to perform additional configuration without user interaction).
+* `foreman` (_String_): The name of the Foreman plugin to be enabled (via `FOREMAN_ENABLED_PLUGINS`).


Can I suggest a bit different approach here:
Have a mandatory folder:
roles/foreman/tasks/features/{{feature_name}}/
And the folder may contain two files: foreman.yaml, smart-proxy.yaml and hammer.yaml. Each file will contain a set of instructions for installing the feature on the foreman machine, smart proxy machine(s) and hammer machines respectively.
We will also create reusable tasks/roles for default actions like installing an RPM, adding smart-proxy config file e.t.c. These tasks will be reused in the yaml files of the feature.
This way we will give the plugin developer full control of the installation procedure and order of tasks execution. It also creates a more explicit installation process.

So the smart proxy deployment would need to read a file from the foreman role? That seems odd to me.

There is also nothing to install, we're in a container here. We can either configure it, or leave it alone.

but that's also not really important right now, the metadata can be defined w/o the actual code being implemented and should have no influence on that

ShimShtein · 2026-02-16T17:17:27Z

docs/feature-metadata.md

+  * **FIXME**: Not implemented, right now we use the same list as Foreman plugins, but needs modification for foreman-tasks and friends
+* `role` (_String_): The name of the Ansible role that should be executed if this feature is enabled.
+* `dependencies` (_Array_ of _String_): List of features that are automatically enabled when the user requests this feature. Usually will point at features with `internal: true`.
+* `requirements` (_Array_ of _String_): List of features that are required but can't be automatically enabled (possible reasons: requires manual configuration, can't be enabled after initial deployment).


I see the question "can the feature be installed automatically" as an internal one, so the developer of a dependent feature should not be aware of this detail. I would like to consolidate the dependencies and requirements nodes. They are both mandatory, and we should leave it to the feature to decide if it has enough information to be installed.

As an enhancement, we may consider adding feature-specific variables for each dependency. For example we may say "I am dependent on the Content feature, but only with Pulp3 as the backend" (assuming that Content feature has a variable backend=pulp3/pulp2).

I agree it would be nice no to expose that distinction, but I don't see a good way to implement those "am I ready" checks.

On the other hand, I think "katello" is the only "weird" feature that we declare as "can't be auto-enabled", and maybe we should ignore that detail for now?

evgeni · 2026-02-24T10:48:37Z

Alternative proposal

During offline discussions about this proposal, we came up with an alternative.

Why

The current proposal is rather tightly coupled to the foreman and foreman_proxy software units,
while in reality we are going into a direction where the "proxy" is not necessarily implemented by foreman_proxy (see the current Pulp deployment).

Nomenclature

Today we use the foreman and foreman_proxy "base" features to determine which designation our deployment has.
Any given deployment can have one or multiple of these, and based on that we deploy different containers and then configure them as needed.
At the same time, it's totally possible that we have a system deployed without foreman and without foreman_proxy: think of an dedicated IoP system, a dedicated database, etc (not that these are possible with todays code, but in future it might be).

Because of that, during the discussion, we started to call the systems "main" and "companion", where the "main" system is the Rails app and multiple companions are offering services to that main system.

Proposal

Based on the distinction between "designation" (main/companion) and "feature", we could define the metadata as follows:

<feature_name>:
  main: <ansible_entrypoint_for_main_deployment>
  companion: <ansible_entrypoint_for_companion_deployment>

This would effectively merge the role entry from the original proposal into the two types.

The intended benefit is that the metadata doesn't expose implementation details like "foreman" and "foreman_proxy" anymore,
at the cost that the "entrypoint" (Ansible role, for example) would need to declare its dependencies ("need foreman_proxy deployed") itself, instead of the metadata implying it.

Longterm vision

Long term we'd like to get to a model where foremanctl can deploy a whole fleet (and not a single system like today), where the user input is something like "deploy system.example.com as a main server with features A, B, C, comp1.example.com as a companion with features B, C and comp2.example.com as a companion with features A, C"

stejskalleos · 2026-02-25T09:11:49Z

docs/feature-metadata.md

+  If `roles/foreman/tasks/feature/{{ foreman_proxy_plugin }}.yaml` exists, it will be executed to perform any plugin-specific setup.
+* `hammer` (_String_): The name of the Hammer plugin to be enabled.
+  * **FIXME**: Not implemented, right now we use the same list as Foreman plugins, but needs modification for foreman-tasks and friends
+* `role` (_String_): The name of the Ansible role that should be executed if this feature is enabled.


What if there will be more roles? Or do we have a rule one feature = one role?

in #280 there is a "entry" role that pulls in the others, but we can also do roles: Array of String instead

stejskalleos · 2026-02-25T09:15:01Z

docs/feature-metadata.md

+
+The following properties are defined:
+* `description` (_String_): A human-readable description of the feature, can be used in documentation/help output.
+* `internal` (_Boolean_): Whether the feature is user visible (shows up in documentation/help) or internal (just to perform additional configuration without user interaction).


In the current foremanctl, what's considered an internal feature?

dynflow in #309 would be one. so far no others.

ehelms · 2026-02-25T16:05:18Z

Alternative proposal

During offline discussions about this proposal, we came up with an alternative.

Why

The current proposal is rather tightly coupled to the foreman and foreman_proxy software units, while in reality we are going into a direction where the "proxy" is not necessarily implemented by foreman_proxy (see the current Pulp deployment).

Nomenclature

Today we use the foreman and foreman_proxy "base" features to determine which designation our deployment has. Any given deployment can have one or multiple of these, and based on that we deploy different containers and then configure them as needed. At the same time, it's totally possible that we have a system deployed without foreman and without foreman_proxy: think of an dedicated IoP system, a dedicated database, etc (not that these are possible with todays code, but in future it might be).

Because of that, during the discussion, we started to call the systems "main" and "companion", where the "main" system is the Rails app and multiple companions are offering services to that main system.

Proposal

Based on the distinction between "designation" (main/companion) and "feature", we could define the metadata as follows:
<feature_name>:
  main: <ansible_entrypoint_for_main_deployment>
  companion: <ansible_entrypoint_for_companion_deployment>
This would effectively merge the role entry from the original proposal into the two types.

The intended benefit is that the metadata doesn't expose implementation details like "foreman" and "foreman_proxy" anymore, at the cost that the "entrypoint" (Ansible role, for example) would need to declare its dependencies ("need foreman_proxy deployed") itself, instead of the metadata implying it.

Longterm vision

Long term we'd like to get to a model where foremanctl can deploy a whole fleet (and not a single system like today), where the user input is something like "deploy system.example.com as a main server with features A, B, C, comp1.example.com as a companion with features B, C and comp2.example.com as a companion with features A, C"

I think I would need to see an example to understand this model better.

The long term vision sounds correct.

I think this where a clear nomenclature will help us. The proposal mentions being tied to software "units". I think it could help engineers and users to tie it to or think in terms of infrastructure components. This allows engineers to have clear conversations with one another, and to follow design patterns that simplify execution. For the user, I believe it helps by allowing them to think about their infrastructure and what's deployed.

Initially I believe our focus should be on simple user interface and understanding, that will upgrade cleanly from rpm-based installations. And where possible, ensure flexibility in our code to allow us to adapt expanded deployment footprints in the future.

Right now, we have two very clear pieces to the setup -- Foreman server and foreman-proxy. Where I am treating foreman-proxy as something more than the smart-proxy.

evgeni · 2026-02-26T15:34:05Z

example based on original proposal:

dynflow:
  internal: true
  foreman_proxy: dynflow

foreman-tasks:
  foreman: foreman-tasks
  hammer: foreman_tasks
  dependencies:
    - dynflow

katello:
  description: Katello
  foreman: katello
  dependencies:
    - foreman-tasks
   
rh_cloud:
  description: Foreman RH Cloud
  foreman: foreman_rh_cloud
  dependencies:
    - katello

remote_execution:
  description: REX
  foreman: foreman_remote_execution
  foreman_proxy: smart_proxy_remote_execution_ssh
  dependencies:
    - dynflow

openscap:
  description: OpenSCAP
  foreman: foreman_openscap
  foreman_proxy: smart_proxy_openscap

iop:
  description: IoP
  role: iop_core

and with the alternative proposal:

dynflow:
  internal: true
  companion: sp_dynflow_role

foreman-tasks:
  main: foreman_tasks_role
  hammer: foreman_tasks
  dependencies:
    - dynflow

katello:
  description: Katello
  main: foreman_katello_role
  dependencies:
    - foreman-tasks
   
rh_cloud:
  description: Foreman RH Cloud
  main: foreman_rh_cloud_role
  dependencies:
    - katello

remote_execution:
  description: REX
  main: foreman_remote_execution_role
  companion: sp_remote_execution_ssh_role
  dependencies:
    - dynflow

openscap:
  description: OpenSCAP
  main: foreman_openscap_role
  companion: sp_openscap_role

iop:
  description: IoP
  main: iop_core

One thing that (IMHO) needs attention: In the original proposal, entries in the foreman and foreman_proxy fields would trigger actions inside the foreman and foreman_proxy Ansible roles that are fully responsible for the setup of the two services. And things that are not direct plugins to those need to be configured via the role entry.
In the alternative proposal every entry is pointing at an Ansible role as we don't know what software type (plugin, standalone) is being deployed.

ehelms · 2026-02-28T13:59:00Z

Is my interpretation correct of the difference:

Different use of nomenclature: main vs. foreman, companion vs foreman_proxy
Specifying a role instead of the name of the dependency

Specifying a role instead of the name seems less flexible, but maps to the way that things were organized inside the puppet modules.

Maybe we combine both to allow greater flexibility and greater explicitness?

foreman_tasks:
  foreman:
    plugin_name: foreman-tasks
  hammer: foreman_tasks
  dependencies:
    - dynflow

katello:
  description: Katello
  foreman:
    plugin_name: katello
    role: katello
  dependencies:
    - dynflow

pr-processor bot added the Not yet reviewed label Feb 12, 2026

evgeni force-pushed the feature-metadata branch 2 times, most recently from d7542f7 to 9e2b1fe Compare February 16, 2026 10:31

evgeni mentioned this pull request Feb 16, 2026

Deploy hammer and smart proxy for development #373

Merged

evgeni commented Feb 16, 2026

View reviewed changes

docs/feature-metadata.md Show resolved Hide resolved

pr-processor bot removed the Not yet reviewed label Feb 16, 2026

evgeni commented Feb 16, 2026

View reviewed changes

evgeni mentioned this pull request Feb 16, 2026

Proxy features: REX #309

Draft

ehelms reviewed Feb 16, 2026

View reviewed changes

docs/feature-metadata.md Outdated Show resolved Hide resolved

ehelms reviewed Feb 16, 2026

View reviewed changes

add first feature metadata draft

53a416c

evgeni force-pushed the feature-metadata branch from 9e2b1fe to 53a416c Compare February 16, 2026 13:31

ShimShtein reviewed Feb 16, 2026

View reviewed changes

stejskalleos reviewed Feb 25, 2026

View reviewed changes

evgeni marked this pull request as ready for review February 25, 2026 09:37


		Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).

		The metadata is a Hash with the feature name as the key and the feature definition as the value.

		@@ -0,0 +1,85 @@
		# Feature Metadata

		Users want to enable abstract features, which means the deployment needs to know how to translate a feature name to a set of changes (configuration files, services, etc).

Conversation

evgeni commented Feb 12, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evgeni commented Feb 24, 2026

Alternative proposal

Why

Nomenclature

Proposal

Longterm vision

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ehelms commented Feb 25, 2026

Alternative proposal

Why

Nomenclature

Proposal

Longterm vision

Uh oh!

evgeni commented Feb 26, 2026

Uh oh!

ehelms commented Feb 28, 2026

Uh oh!

Reviewers