diff --git a/.github/agents/test-specialist.md b/.github/agents/test-specialist.md index f3666d010e..e980d78d76 100644 --- a/.github/agents/test-specialist.md +++ b/.github/agents/test-specialist.md @@ -13,6 +13,7 @@ You are a testing specialist focused on improving code quality through comprehen Always include clear test descriptions and use appropriate testing patterns for the language and framework. + ## Testing Patterns for flexmeasures FlexMeasures uses pytest with two main fixture patterns for database management: @@ -172,6 +173,22 @@ When pre-commit hooks fail: - Ensure all tests pass before asking for a review - Update these agent instructions with learnings from each assignment +## Environment Setup + +**IMPORTANT**: Before running tests, ensure your environment is properly configured. +Follow the standardized setup instructions in: + +- **`.github/workflows/copilot-setup-steps.yml`** (owned by Tooling & CI Specialist) + +This file contains all necessary steps for: + +- System dependencies (PostgreSQL, Redis) +- Python dependencies +- Database setup +- Environment variables + +If setup steps fail or are unclear, escalate to the Tooling & CI Specialist. + ## Running Tests in FlexMeasures Dev Environment ### Critical Requirement: Actually Run Tests @@ -262,6 +279,44 @@ flask db current - **Assertions**: Use descriptive assertion messages for failures - **Mocking**: Use pytest fixtures and mocking when testing external dependencies +## Test-Driven Bug Fixing (CRITICAL PATTERN) + +When fixing failing tests, ALWAYS follow this test-driven approach: + +### Step 1: Reproduce the Failure FIRST + +- **Run the actual test** to see it fail (don't just read code) +- **Capture the exact error message** and failure output +- **Understand the failure mode**: What was expected vs. what happened? + +### Step 2: Debug to Understand Root Cause + +- **Use debugger tools**, not just code inspection: + - `pytest --pdb` to drop into debugger on failure + - Add `import pdb; pdb.set_trace()` at strategic points + - Use print statements to trace execution flow +- **Trace the actual execution path** through the code +- **Look for MULTIPLE bugs**, not just the obvious one + - Example: Session learned that a test failure involved BOTH an API bug AND a test bug + +### Step 3: Apply Fix(es) + +- Fix all identified bugs (API, test, or both) +- Make atomic commits (separate production code from test code changes) +- Document WHY the bug existed and HOW the fix works + +### Step 4: Verify the Fix + +- **Re-run the specific test** to confirm it now passes +- **Check for regressions**: Run related tests or entire test suite +- **Don't claim "tests pass" without actually running them** + +### Step 5: Update Agent Instructions + +- Document the lesson learned in this file +- What pattern or pitfall should be remembered? +- What verification step was missing? + ## Commit Discipline for Test Changes When updating tests or this agent file: diff --git a/.github/agents/tooling-ci-specialist.md b/.github/agents/tooling-ci-specialist.md index de9fc88a09..88d9a46420 100644 --- a/.github/agents/tooling-ci-specialist.md +++ b/.github/agents/tooling-ci-specialist.md @@ -15,6 +15,7 @@ Keep FlexMeasures automation reliable and maintainable by reviewing GitHub Actio - GitHub Actions workflows (`.github/workflows/`) - Pre-commit configuration (`.pre-commit-config.yaml`) +- Agent environment setup (`.github/workflows/copilot-setup-steps.yml`) - Linter configurations (flake8, black, mypy) - Build and deployment scripts - CI matrix strategy (Python versions, services) @@ -74,6 +75,45 @@ Keep FlexMeasures automation reliable and maintainable by reviewing GitHub Actio - [ ] **Fail-fast**: Usually false for comprehensive testing - [ ] **Coverage**: One Python version runs coverage +### Agent Environment Setup + +File: **`.github/workflows/copilot-setup-steps.yml`** + +This file defines standardized environment setup for GitHub Copilot agents. When reviewing or updating: + +- [ ] **System dependencies**: Are all required packages installed? + - PostgreSQL client libraries (`libpq-dev`) + - Redis server + - Other system tools + +- [ ] **Python environment**: + - Is Python version appropriate? (Default: 3.11) + - Are dependencies installed correctly? (`pip-sync`, `pip install -e .`) + - Is pip-tools version pinned? + +- [ ] **Database setup**: + - Is PostgreSQL service started? + - Are test user and database created correctly? + - Are permissions granted? (`CREATEDB` privilege) + - Are extensions loaded? (`ci/load-psql-extensions.sql`) + +- [ ] **Environment variables**: + - `FLEXMEASURES_ENV=testing` + - `SQLALCHEMY_DATABASE_URI` (PostgreSQL connection string) + - `FLEXMEASURES_REDIS_URL` (Redis connection string) + +- [ ] **Documentation**: + - Are usage notes clear and accurate? + - Are common issues and solutions documented? + - Are testing commands documented? + +**IMPORTANT**: When this file is updated, verify it actually works: + +1. Follow the setup steps in a clean environment +2. Run tests to confirm environment is functional +3. Document any issues or unclear steps +4. Update the file based on learnings + ## Domain Knowledge ### FlexMeasures CI Infrastructure diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml new file mode 100644 index 0000000000..24f63d0db1 --- /dev/null +++ b/.github/workflows/copilot-setup-steps.yml @@ -0,0 +1,118 @@ +# GitHub Copilot Agent Environment Setup +# This file defines the standardized environment setup for GitHub Copilot agents +# working on the FlexMeasures repository. +# +# Reference: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment + +name: FlexMeasures Development Environment Setup + +# Define the steps that Copilot agents should execute to set up the development environment +setup-steps: + - name: Install system dependencies + run: | + # Update package list + sudo apt-get update + + # Install PostgreSQL client libraries (required for psycopg2) + sudo apt-get install -y libpq-dev + + # Install Redis (used for job queuing) + sudo apt-get install -y redis-server + + # Start Redis service + sudo service redis-server start + + - name: Set Python version + run: | + # FlexMeasures supports Python 3.9-3.12 + # Use Python 3.11 as the default for agent development + python --version || python3 --version + + - name: Install pip-tools + run: | + # Install pip-tools for dependency management + pip3 install -q "pip-tools>=7.2" + + - name: Install Python dependencies for testing + run: | + # Get Python version (major.minor) + PYV=$(python -c "import sys;t='{v[0]}.{v[1]}'.format(v=list(sys.version_info[:2]));sys.stdout.write(t)") + + # Install pinned test and app dependencies + pip-sync requirements/${PYV}/app.txt requirements/${PYV}/test.txt + + # Install FlexMeasures in editable mode + pip install -e . + + - name: Install pre-commit hooks + run: | + # Install pre-commit for code quality checks + pip install pre-commit + + # Install the pre-commit hooks + pre-commit install + + - name: Setup PostgreSQL for testing + run: | + # Install PostgreSQL if not already available + sudo apt-get install -y postgresql postgresql-contrib + + # Start PostgreSQL service + sudo service postgresql start + + # Drop existing database and user if they exist (for clean setup) + sudo -u postgres psql -c "DROP DATABASE IF EXISTS flexmeasures_test;" + sudo -u postgres psql -c "DROP USER IF EXISTS flexmeasures_test;" + + # Create test database and user with correct permissions + sudo -u postgres psql -c "CREATE USER flexmeasures_test WITH PASSWORD 'flexmeasures_test';" + sudo -u postgres psql -c "CREATE DATABASE flexmeasures_test OWNER flexmeasures_test;" + sudo -u postgres psql -c "ALTER USER flexmeasures_test CREATEDB;" + + # Load PostgreSQL extensions (timescaledb, etc.) + sudo -u postgres psql -U flexmeasures_test -d flexmeasures_test -f ci/load-psql-extensions.sql || echo "Extensions loaded or not available" + + - name: Set environment variables + run: | + # Set FlexMeasures environment to testing + echo "FLEXMEASURES_ENV=testing" >> $GITHUB_ENV + + # Set database URL for tests (using PostgreSQL) + echo "SQLALCHEMY_DATABASE_URI=postgresql://flexmeasures_test:flexmeasures_test@localhost/flexmeasures_test" >> $GITHUB_ENV + + # Set Redis URL for job queuing + echo "FLEXMEASURES_REDIS_URL=redis://localhost:6379/0" >> $GITHUB_ENV + +# Notes for agents: +# +# 1. After running these setup steps, you can run tests with: +# - pytest # Run all tests +# - pytest path/to/test_file.py # Run specific test file +# - pytest -k test_name # Run specific test by name +# - pytest -x # Stop on first failure +# - pytest -v # Verbose output +# +# 2. To run pre-commit hooks manually: +# - pre-commit run --all-files # Run all hooks on all files +# - pre-commit run flake8 # Run specific hook +# +# 3. Database management: +# - flask db upgrade # Run database migrations +# - flask db current # Show current migration version +# +# 4. The Makefile provides shortcuts: +# - make install-for-test # Install test dependencies +# - make test # Run all tests +# - make generate-openapi # Generate OpenAPI specs +# +# 5. Key testing fixtures (defined in conftest.py): +# - db: Database session fixture +# - app: Flask app fixture +# - fresh_db: Fresh database for isolated tests +# - setup_roles_users: User and role fixtures +# +# 6. Common issues and solutions: +# - If PostgreSQL connection fails: Check service is running with `sudo service postgresql status` +# - If Redis connection fails: Check service is running with `sudo service redis-server status` +# - If tests fail with import errors: Ensure FlexMeasures is installed with `pip install -e .` +# - If pre-commit hooks fail: Run `pre-commit run --all-files` to see specific issues diff --git a/documentation/changelog.rst b/documentation/changelog.rst index e349f6d30f..f0666ce402 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -50,6 +50,7 @@ Infrastructure / Support * Improve reported test coverage by including doctests and bringing back CLI tests to GitHub Actions [see `PR #1914 `_ and `PR #778 `_, respectively] * Fix README badges [see `PR #1913 `_] * Allow seeing complete datetimes in the audit log [see `PR #1949 `_] +* Add standardized GitHub Copilot agent environment setup file for consistent development environment across agent sessions [see `PR #1962 `_] Bugfixes -----------