Add POST endpoints for creating annotations on accounts, assets, and sensors

.github/agents/api-backward-compatibility-specialist.md

@@ -50,6 +50,103 @@ Protect FlexMeasures users and integrators by ensuring API changes are backwards
 - [ ] **Type changes**: Field type changes are breaking
 - [ ] **Validation**: Stricter validation is breaking, looser is safe
 - [ ] **Marshmallow schemas**: Check schema version compatibility
+- [ ] **Response schema completeness**: Verify all response schemas include required fields (see Response Schema Patterns below)
+
+#### Response Schema Patterns
+
+**Always use separate input and output schemas for API endpoints**
+
+Input schemas validate request data; output schemas control response data. These should be separate classes even if they share fields.
+
+**Checklist for Response Schema Completeness**:
+
+- [ ] **ID field**: Response must include `id` field for created/updated resources
+- [ ] **Source field**: If resource has a source, include it in response
+- [ ] **Audit fields**: Consider including `created_at`, `updated_at` if relevant
+- [ ] **All identifying fields**: Include fields clients need to reference the resource
+- [ ] **Idempotency support**: Provide enough data for clients to detect duplicates
+
+**Session 2026-02-10 Case Study** (Annotation API):
+
+**Problem**: Initial annotation API used single schema for input and output:
+
+```python
+class AnnotationSchema(Schema):
+    content = fields.String(required=True)
+    start = AwareDateTimeField(required=True, format="iso")
+    # Missing: id field in output
+```
+
+**Issue**: Clients couldn't retrieve the `id` of created annotations, breaking idempotency checks.
+
+**Wrong Fix**: Separate input and output schemas:
+
+```python
+class AnnotationSchema(Schema):
+    """Input schema - validates request data"""
+    content = fields.String(required=True)
+    start = AwareDateTimeField(required=True, format="iso")
+
+class AnnotationResponseSchema(Schema):
+    """Output schema - includes all data clients need"""
+    id = fields.Integer(required=True)
+    content = fields.String(required=True)
+    start = AwareDateTimeField(required=True, format="iso")
+```
+
+**Right Fix**:
+
+```python
+class AnnotationResponseSchema(Schema):
+    """One schema - validates request data and includes all data clients need.
+
+    Please note:
+    - the use of `dump_only`
+    - metadata description and example(s) must always be included.
+    - we prefer single-word data keys over snake_case or kebab-case data keys.
+    """
+    id = fields.Int(
+        dump_only=True,
+        metadata=dict(
+            description="The annotation's ID, which is automatically assigned.",
+            example=19,
+        ),
+    )
+    content = fields.Str(
+        required=True,
+        validate=Length(max=1024),
+        metadata={
+            "description": "Text content of the annotation (max 1024 characters).",
+            "examples": [
+                "Server maintenance",
+                "Installation upgrade",
+                "Operation Main Strike",
+            ],
+        },
+    )
+    start = AwareDateTimeField(
+        required=True,
+        format="iso",
+        metadata={
+            "description": "Start time in ISO 8601 format.",
+            "example": "2026-02-11T17:52:03+01:00",
+        },
+    )
+    source_id = fields.Int(
+        data_key="source",
+        dump_only=True,
+        metadata=dict(
+            description="The annotation's data source ID, which usually corresponds to a user (it is not the user ID, though).",
+            example=21,
+        ),
+    )
+ ```
+
+**Why This Matters**:
+- Clients need `id` to detect if they've already created this annotation
+- Clients need `id` to update or delete the annotation later
+- Missing fields force clients to make additional API calls
+- Breaks RESTful conventions (POST should return created resource)
 
 #### Parameter Format Consistency
 
@@ -182,6 +279,120 @@ Configuration: `FLEXMEASURES_PLUGINS`
 
 Known plugins: flexmeasures-client, flexmeasures-weather, flexmeasures-entsoe
 
+### Idempotency Detection Patterns
+
+**Never rely on `obj.id is None` to detect if object is new**
+
+SQLAlchemy may not assign IDs until commit, and the pattern is unreliable.
+
+**Anti-pattern** (Session 2026-02-10):
+```python
+annotation = get_or_create_annotation(...)
+is_new = annotation.id is None  # ❌ Unreliable!
+if is_new:
+    return success_response, 201
+else:
+    return success_response, 200
+```
+
+**Problem**: `annotation.id` might be `None` even for existing objects before flush/commit.
+
+**Correct pattern**: Make helper functions return explicit indicators:
+```python
+def get_or_create_annotation(...):
+    existing = db.session.query(Annotation).filter_by(...).first()
+    if existing:
+        return existing, False  # (object, was_created)
+    new_obj = Annotation(...)
+    db.session.add(new_obj)
+    return new_obj, True
+
+# In endpoint:
+annotation, was_created = get_or_create_annotation(...)
+status_code = 201 if was_created else 200
+```
+
+**Why This Matters**:
+- Idempotency is critical for API reliability
+- Wrong status codes break client logic
+- Clients depend on 201 vs 200 to track resource creation
+
+### Error Handling Patterns
+
+**Catch specific exceptions, not bare `Exception`**
+
+**Anti-pattern**:
+```python
+try:
+    annotation = get_or_create_annotation(...)
+except Exception as e:  # ❌ Too broad!
+    return error_response("Failed to create annotation")
+```
+
+**Problems**:
+- Catches programming errors (AttributeError, TypeError, etc.)
+- Hides bugs that should fail loudly
+- Makes debugging difficult
+
+**Correct pattern**:
+```python
+from sqlalchemy.exc import SQLAlchemyError
+
+try:
+    annotation = get_or_create_annotation(...)
+except SQLAlchemyError as e:  # ✅ Specific database errors
+    db.session.rollback()
+    return error_response("Database error creating annotation", 500)
+except ValueError as e:  # ✅ Expected validation errors
+    return error_response(str(e), 400)
+# Let other exceptions propagate - they indicate bugs
+```
+
+**When to catch what**:
+- `SQLAlchemyError`: Database errors (connection, integrity, etc.)
+- `ValueError`: Expected validation failures
+- `KeyError`: Missing required data
+- Don't catch: `AttributeError`, `TypeError`, `NameError` (these are bugs)
+
+### Experimental API Documentation
+
+**Endpoints in `/api/dev/` must warn users about instability**
+
+**Required elements**:
+
+1. **Docstring warning**:
+```python
+@annotations_bp.route("/annotations", methods=["POST"])
+def create_annotation():
+    """Create a new annotation.
+
+    .. warning::
+        This endpoint is experimental and may change without notice.
+        It is not subject to semantic versioning guarantees.
+    """
+```
+
+2. **Response header** (if applicable):
+```python
+response.headers["X-API-Stability"] = "experimental"
+```
+
+3. **OpenAPI metadata**:
+```yaml
+/api/dev/annotations:
+  post:
+    tags:
+      - experimental
+    description: |
+      ⚠️ **Experimental API** - This endpoint may change without notice.
+```
+
+**Why This Matters**:
+- Users integrating with `/api/dev/` need clear expectations
+- Protects maintainers' ability to iterate quickly
+- Prevents users from depending on unstable contracts
+- Documents the migration path to stable API
+
 ### Related Files
 
 - API: `flexmeasures/api/`

.github/agents/architecture-domain-specialist.md

@@ -40,6 +40,7 @@ This agent owns the integrity of models (e.g. assets, sensors, data sources, sch
 - [ ] **Account ownership**: Check that all assets have `account_id` set correctly
 - [ ] **Sensor-Asset binding**: Validate sensors are properly linked to assets
 - [ ] **TimedBelief structure**: Ensure (event_start, belief_time, source, cumulative_probability) integrity
+- [ ] **Annotation relationships**: Verify many-to-many associations use relationship append pattern
 
 ### Flex-context & flex-model
 
@@ -63,6 +64,7 @@ This agent owns the integrity of models (e.g. assets, sensors, data sources, sch
 - [ ] **No quick hacks**: Push back on changes that erode model clarity for short-term gains
 - [ ] **Separation of concerns**: Validate, process, and persist should be distinct steps
 - [ ] **Multi-tenancy**: Ensure account-level access control is maintained
+- [ ] **Idempotency**: API endpoints should use get-or-create patterns with proper tuple returns for status detection
 
 ### Schema-Code Consistency
 
@@ -233,6 +235,28 @@ fields_to_remove = ["as_job"]  # ❌ Wrong format
 - **Location**: `/flexmeasures/data/models/data_sources.py`
 - **Purpose**: Forecasters and reporters subclass `DataGenerator` to couple configured instances to unique data sources (schedulers are not yet subclassing `DataGenerator`)
 
+#### Annotation
+- **Location**: `flexmeasures/data/models/annotations.py`
+- **Purpose**: Independent entities for metadata about other domain objects (assets, sensors, accounts)
+- **Relationships**: Many-to-many with GenericAsset, Sensor, Account (via association tables)
+- **Key fields**: `id`, `content`, `type`, `start`, `end`, `source_id`
+- **Pattern**: Use `get_or_create_annotation()` for idempotency
+  - Returns `(annotation, is_new)` tuple
+  - `is_new=True` for created, `is_new=False` for existing
+  - Enables proper HTTP status codes (201 vs 200)
+- **Association pattern**: Use SQLAlchemy relationship append
+  ```python
+  # ✅ Correct: Use relationship append
+  entity.annotations.append(annotation)
+
+  # ❌ Wrong: Manual join table manipulation
+  # Don't create association table entries directly
+  ```
+- **Permission model**: Annotations are independent entities
+  - Adding annotation to entity requires "update" permission on entity
+  - Not "create-children" (that's for owned hierarchies like asset→sensor)
+  - Rationale: Many-to-many relationship, annotation exists independently
+
 ### Critical Invariants
 
 1. **Acyclic Asset Trees**
@@ -296,6 +320,83 @@ fields_to_remove = ["as_job"]  # ❌ Wrong format
 - **Tight coupling**: API/CLI should not import from each other
 - **Bypassing services**: Direct model access from API/CLI
 - **Quick hacks**: Temporary solutions that become permanent
+- **Manual join table manipulation**: Use SQLAlchemy relationship methods, not direct association table inserts
+- **Wrong permission model**: Use "update" for annotations, not "create-children" (which is for owned hierarchies)
+- **Idempotency without detection**: get-or-create functions should return `(entity, is_new)` tuple
+
+### Annotation API Pattern (Session 2026-02-10)
+
+When implementing POST endpoints for adding annotations to domain entities:
+
+#### 1. Idempotency Pattern
+```python
+# ✅ Correct: get_or_create returns tuple
+annotation, is_new = get_or_create_annotation(
+    content=...,
+    type=...,
+    source_id=...,
+    # ... other fields
+)
+
+# Return appropriate HTTP status
+if is_new:
+    return make_response({...}, 201)  # Created
+else:
+    return make_response({...}, 200)  # OK (already exists)
+```
+
+#### 2. Relationship Management Pattern
+```python
+# ✅ Correct: Use SQLAlchemy relationship append
+entity.annotations.append(annotation)
+db.session.commit()
+
+# ❌ Wrong: Manual join table manipulation
+# Don't create rows in association tables directly
+```
+
+#### 3. Permission Pattern
+For many-to-many relationships like annotations:
+- **Use "update" permission** to add annotation to entity
+- **Not "create-children"** (that's for owned hierarchies like GenericAsset→Sensor)
+- **Rationale**: Annotation is independent entity, can be associated with multiple entities
+
+#### 4. API Endpoint Structure
+```python
+class AnnotationAPI(FlaskView):
+    @route("/<resource_id>/annotations", methods=["POST"])
+    @permission_required_for_context("update", ctx_arg_name="entity")
+    def post(self, resource_id: int):
+        # 1. Load and validate entity
+        entity = get_entity_or_abort(resource_id)
+
+        # 2. Validate annotation data (Marshmallow schema)
+        annotation_data = AnnotationSchema().load(request.json)
+
+        # 3. Get or create annotation (idempotency)
+        annotation, is_new = get_or_create_annotation(**annotation_data)
+
+        # 4. Associate with entity (relationship append)
+        entity.annotations.append(annotation)
+        db.session.commit()
+
+        # 5. Return appropriate status
+        status_code = 201 if is_new else 200
+        return make_response(AnnotationSchema().dump(annotation), status_code)
+```
+
+#### 5. Review Checklist for Annotation Endpoints
+- [ ] Does `get_or_create_annotation()` return `(annotation, is_new)` tuple?
+- [ ] Does endpoint return 201 for new, 200 for existing?
+- [ ] Does code use `entity.annotations.append()` not manual join table?
+- [ ] Does endpoint require "update" permission not "create-children"?
+- [ ] Is annotation data validated with Marshmallow schema?
+- [ ] Is the association committed to database?
+
+**Related Files**:
+- Model: `flexmeasures/data/models/annotations.py`
+- API: `flexmeasures/api/v3_0/assets.py`, `flexmeasures/api/v3_0/sensors.py`
+- Schema: `flexmeasures/data/schemas/annotations.py`
 
 ### Related Files