Move config DataModel to Pydantic

[Julfried]

I think it would make sense to move the Datamodel for all the configurations to pydantic. This way would simplify a lot of the code and would make reading the config file way more robust.

[Julfried]

@MaStr What do you think of this? I would start to work in this then :)

[MaStr]

I didn't understood it, tbh

What are the benefits?
How can we achieve backwards compatibility?

[MaStr]

I pushed some cleanups in #145 , which solves the 3 million different solutions.
That will help.

I'm afraid of adding another big change, while I'm still tinkering on fixing side effects.

[Julfried]

@MaStr - The Pydantic approach would primarily be about validation, not changing how we work with the config.

Benefits:

• Type validation happens automatically at load time at a single place in the code
• Default values are handled consistently across the whole codebase
• Configuration errors are caught early with clear messages that indicates what is wrong in the config
• Automatic generation of the yaml documentation

I understand your concern about big changes while you're still fixing side effects. This would be a gradual approach - we can start with a small validation layer that doesn't change existing code flow.

Here's a simple, generic example of how Pydantic works:

from pydantic import BaseModel

# Define what we expect
class ServerConfig(BaseModel):
    host: str
    port: int = 8080  # Default value
    debug: bool = False

# Validate input
config_data = {"host": "localhost", "port": "5000"}  # Note: port is a string and debug is not provided
config = ServerConfig(**config_data)  # Pydantic converts port to int automatically

# The dataclass can be dumped as a dict ==> backwards compatible
config_dict = config.model_dump()
print(f"Connecting to {config_dict['host']}:{config_dict['port']}. Debug mode: {config_dict['debug']}")

config_data = {"host": "localhost", "port": 6000, "debug": "Wrong"}  # Note: debug is now a string
config = ServerConfig(**config_data)  # This raises a validation error

This is a long-term goal that can be implemented incrementally, without disrupting your the work on bug fixes. Pydantic models support converting them into a dict so it would also be compatible for now.

Maybe I should start with a small proof of concept in a separate branch, by defining BaseModels for every major module (inverter, forecastsolar, forecastconsumption and dynamictariff)

[MaStr]

Ah! I did something similar codewise here https://github.com/MaStr/batcontrol_wp-feedin/blob/main/wp_em_adjustment.py

That makes sense to me.

Thank you for the extra mile in explaining that.

[Julfried]

Your welcome ;). This is already a great idea, and exactly what I would implement in every major module like inverter, forecastsolar , etc. I would just change that this class inherits from pydantics Basemodel. That way pydantic would handle all the validation of the yaml config for you in the background. I the future there might be the need to have a documentation for the yaml file specification, so pydantic would also allow to automatically generate that based on the BaseModels defined in the code :)