My CI is not seeing my environment variables!

[thorwhalen]

My tests are failing because the code doesn't see my environment variables, though I do have them in my "repository secrets".

Concrete example:

This part fails:

      - name: Pytest Validation
        uses: i2mint/isee/actions/pytest-validation@master
        with:
          root-dir: ${{ env.PROJECT_NAME }}
          paths-to-ignore: examples,scrap

where (currently) the `` contents can be found here

[thorwhalen]

tldr;

Add this to the part of the CI that fails:

        env:
          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}

details

To ensure that your GitHub Actions workflow can access the KAGGLE_KEY environment variable that you've stored in your repository secrets, you need to explicitly pass this secret to the workflow. Secrets are not passed to the runner environment by default for security reasons.

Here's how you can modify your ci.yml file to include the KAGGLE_KEY environment variable:

      - name: Pytest Validation
        uses: i2mint/isee/actions/pytest-validation@master
        with:
          root-dir: ${{ env.PROJECT_NAME }}
          paths-to-ignore: examples,scrap
        env:
          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}

In this snippet, the env section under the Pytest Validation step is used to set environment variables for that step. Here, KAGGLE_KEY is set to the value of the secret KAGGLE_KEY from your repository's secrets.

Remember, for this to work:

1. The secret KAGGLE_KEY must be correctly set in your repository's settings under the 'Secrets' section.

2. The GitHub Actions runner needs to have the appropriate permissions to access repository secrets. This is usually the case by default, but if you're using a self-hosted runner or have modified the default permissions, you may need to adjust them.

3. Ensure that your code is correctly configured to use the KAGGLE_KEY environment variable.

By making this change, your GitHub Actions workflow should be able to access the KAGGLE_KEY environment variable and use it during the Pytest Validation step.

[thorwhalen]

Still, the solutions above (from chatGPT) are not satisfactory since they all require a list of common environment variables that would cover all projects. As soon as the list is not exhaustive (and it will never be for the general population), one would need to edit the template.

Instead, I propose that instead of:

    env:
      COMMON_ENV_VARS: 'KAGGLE_KEY,ANOTHER_VAR,SOME_OTHER_VAR'

we have something like:

    env:
     ENV_VARS: ${{ env.ENV_VARS }}

so that the user can specify the list of keys in a place that's separate from the template itself, which can now be left un-edited.

We could also consider using a script/function that would fetch this list automatically/dynamically. Are there any security concerns there?

def get_repo_secret_keys(owner, repo, token=None):
    """Gets list of secret keys for a given repo."""
    import json
    import urllib.request

    url = f"https://api.github.com/repos/{owner}/{repo}/actions/secrets"

    if token is None:
        token = os.environ.get("GITHUB_TOKEN", None)
    headers = {
        "Accept": "application/vnd.github.v3+json"
    }
    if token is not None:
        headers["Authorization"] = f"token {token}"

    req = urllib.request.Request(url, headers=headers)

    with urllib.request.urlopen(req) as response:
        result = json.load(response)
        secret_keys = [secret['name'] for secret in result.get('secrets', [])]
        return secret_keys

[thorwhalen]

Your approach of using an environment variable to specify the list of secret keys (ENV_VARS) is a good idea for maintaining flexibility across different projects without altering the template. This way, you can set ENV_VARS in each project's GitHub Actions environment or in the workflow dispatch/event inputs, thereby keeping the template generic.

Regarding the use of a script or function to fetch the list of secret keys dynamically, there are a few security and practical considerations:

1. Exposure of Sensitive Information: Automatically fetching and setting all secrets as environment variables might inadvertently expose sensitive information. It's crucial to ensure that only necessary secrets are made available to the workflow.

2. Access Token Permissions: The script requires a GitHub token with sufficient permissions (typically admin rights to the repository) to access the secrets. This token must be handled securely.

3. Token Storage: If the token is stored as a secret and used within the same workflow to fetch other secrets, it creates a recursive dependency, which can be complex to manage securely.

4. Execution Context: The script needs to be executed in a context where it has access to the necessary credentials to authenticate with the GitHub API. This is typically before the main actions that require these environment variables.

5. Rate Limiting and Error Handling: When interacting with the GitHub API, consider rate limiting and implement robust error handling to manage potential issues during the API call.

6. Workflow Complexity: Adding this level of automation increases the complexity of the workflow. It's important to balance the need for automation with the maintainability and readability of the workflow.

Your modified Python function looks suitable for fetching the list of secret keys. Remember to handle the GITHUB_TOKEN securely and ensure that it's only used in contexts where its exposure is controlled. The script should ideally be executed in a step before any other steps that require those environment variables.

In conclusion, while your approach adds flexibility, it also introduces complexity and potential security considerations. It's crucial to weigh these factors against the benefits of automation in your specific context.

[thorwhalen]

See: thorwhalen/haggle#1

[valentin-feron]

If the intent is to use env.SOMETHING in reusable ci scripts like

isee/actions/pytest-validation/action.yml

Lines 1 to 42 in e3f6932

	name: "Pytest Validation"
	description: "Validate source code running pytest"
	inputs:
	root-dir:
	description: "Root directory to validate. Defaults to the current directory."
	required: false
	default: "."
	paths-to-ignore:
	description: "Folders and or files to ignore. Provide a comma separated list of
	paths from the current directory. Any file under these paths will be ignored.
	Example: 'examples,core/scrap,setup.py'"
	required: false
	skip-doctests:
	description: "Skip doctests"
	required: false
	pytest-args:
	description: "Additional arguments to pass to pytest"
	required: false
	default: "-v"
	runs:
	using: 'composite'
	steps:
	- name: Install Dependencies
	uses: i2mint/isee/actions/install-packages@master
	with:
	pypi-packages: 'pytest'
	- name: Validate Source Code
	shell: bash
	run: |
	options="${{ inputs.pytest-args }}"
	if [ -n "${{ inputs.paths-to-ignore }}" ]; then
	IFS=',' read -ra paths <<< "${{ inputs.paths-to-ignore }}"
	for path in "${paths[@]}"
	do
	options+=" --ignore=${{ inputs.root-dir }}/$path"
	done
	fi
	if [ "${{ inputs.skip-doctests }}" != "true" ]; then
	options+=" --doctest-modules"
	fi
	pytest $options ${{ inputs.root-dir }}

I don't think we should. We should always use inputs so the scripts do not depend on any environment configuration. Unless there is a case I am missing that would justify such a behavior. In that case, please share it (I don't see any ref to KAGGLE_KEY in the latest isee)

[thorwhalen]

Let me clarify.

Code sometimes depends on the existence of environment configuration -- for example, secret tokens.
These usually require manual user input: Either the install instructions tell the user to acquire and specify this information via environment variables, or/and ask the user for this information if/when missing.

Of course, there's pros and cons to this way of doing things, but we don't really have the choice here, if we're thinking of adapting to a common practice.

The problem is: Given code whose tests depend on environment variables, how do we communicate these to our Github actions processes.

It seems you're saying "don't depend on environment variables", which is not a solution to the problem above, just an avoidance of it. The XY problem argument (if that's what's happening here) doesn't apply as I see it.
Code will depend on environmental variables.